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WEB-ENABLED  EXERCISE  GENERATION  TOOL  FOR  BATTLE  COMMAND  TRAINING 


EXECUTIVE  SUMMARY 


Research  Requirement: 

The  rate  of  change  in  the  modern  operational  environment  requires  U.S.  Army  trainers  to 
deliver  effective  training  in  less  time  than  ever  before.  Among  the  changing  environmental 
conditions  are  enemy  tactics,  new  vehicles  and  equipment,  updated  command,  control,  and 
communications  systems,  unit  composition,  and  even  unit  structure.  Each  of  these  changes 
produces  new  requirements  for  structured  training  such  that  knowledge  acquired  from  the 
operational  environment  can  be  transformed  into  skilled  behavior.  For  Soldiers,  most  learning 
occurs  on  the  job  and  is  specific  to  the  unit  in  which  the  Soldier  is  operating,  so  trainers  keeping 
pace  with  environmental  change  must  be  able  to  quickly  develop  tailored  training  in  order  to 
foster  human  capital  in  their  unit. 

Research  and  development  to  support  Soldier  training  must  explore  not  only  advanced 
learning  environments  and  instructional  strategies  but  also  advanced  training  development 
processes.  Advanced  training  development  processes  may  enable  the  rapid  generation  of  training 
activities  that  are  responsive  to  immediate  training  needs.  This  final  report  documents  a  Phase  II 
Small  Business  Technology  Transfer  (STTR)  project  to  research  and  develop  an  integrated 
platform  of  technologies  (the  Training  Assistant  [TA])  to  enable  “one-stop”  creation,  delivery, 
and  management  of  Web-enabled  multimedia  training  exercises.  The  report  begins  by 
summarizing  the  Phase  I  research  and  development,  then  describes  the  Phase  II  work  in  detail. 
The  report  closes  with  conclusions  and  recommendations  for  future  research  based  on  initial 
testing  and  evaluation  of  the  Phase  II  product. 

Procedure: 

The  Phase  I  research  analyzed  the  Army  training  process,  identified  methods  for  relieving 
the  constraints  on  rapid,  contextualized  training  development,  and  developed  these  methods  into 
a  prototype  TA  capability  for  feasibility  analysis.  The  Phase  II  work  detailed  in  this  report 
consisted  of  behavioral  research  to  examine  the  target  user  characteristics  and  the  likely-use 
environment  to  ensure  that  the  TA  would  optimally  support  the  flexible  development  of 
decision-making  exercises  (DMX).  In  addition,  the  work  focused  on  development  a  tool  to 
develop  new  training  in  a  way  that  is  easy  to  use  and  integrated  into  a  larger  instructional  context 
that  individualizes  training.  Phase  II  also  involved  the  development  of  a  working  TA  prototype 
and  usability  testing  of  the  TA  prototype. 

Findings: 

We  found  that  trainers  would  use  the  TA  to  rapidly  develop  foundational  interactive 
multimedia  exercises  for  individuals  and/or  small  groups.  The  research  further  found  that  trainers 
may  approach  the  creation  of  exercises  with  a  particular  doctrinal  task  they  wish  to  train  (e.g., 
report  tactical  information)  and  may  approach  training  with  a  general  competency  they  wish  to 
develop  (e.g.,  communications  skills).  Additionally,  the  results  indicated  that  trainers  are  familiar 
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with  mainstream  software  applications  (especially  Microsoft®  Word  and  PowerPoint),  Internet 
browsers  (especially  Microsoft®  Internet  Explorer),  search  engines  (especially  Google), 
knowledge  management  portals  (especially  Wikipedia,  Questia,  and  Army-specific  portals),  and 
computer-based  instruction,  and  that  few  trainers  have  formal  education  in  instructional  design. 
Finally,  the  results  indicated  that  the  design  of  the  TA  must  resonate  with  trainers’  understanding 
of  doctrine  but  be  flexible  enough  to  address  necessary  deviations  from  doctrinal  practices,  such 
as  “train-as-you- fight.” 

With  regard  to  familiarity  with  computers,  software  applications,  and  other  technology, 
trainees  have  similar  characteristics  as  trainers.  Trainees  would  use  the  TA  to  access  and  conduct 
training  developed  by  their  unit  trainers  or  instructors.  Trainees  also  may  access  the  TA  to 
conduct  self-development  using  training  residing  in  the  TA  archives.  The  training  created  in  the 
TA  must  be  easy  for  trainees  to  search,  locate,  and  access.  The  exercises  must  be  engaging,  and 
performance  feedback  must  be  immediate  and  perceived  as  fair. 

To  meet  the  needs  of  both  trainers  and  trainees,  the  TA  was  designed  to  serve  as  a 
training  delivery  tool,  an  authoring  tool,  and  a  learning  management  system  (LMS).  The 
functions  of  the  tool  are  determined  by  a  user’s  level  of  access  (e.g.,  student,  trainer, 
administrator).  Structured  training  templates,  generic  base  content,  and  an  integrated  search  were 
the  three  main  features  incorporated  into  the  tool  to  increase  the  speed  with  which  trainers  can 
create  quality  training. 

User  testing  of  the  TA  took  place  at  Fort  Hood  and  Fort  Carson,  and  included  29 
Soldiers.  Results  indicated  that  test  participants  generally  found  the  TA  to  be  easy  to  use  (46- 
86%,  depending  on  the  feature  assessed)  with  the  majority  reporting  that  they  could  and  would 
use  it  (75%  and  66%  respectively).  Usability  enhancements  were  identified,  and  consistent  with 
the  purpose  of  alpha  testing,  users  also  identified  a  number  of  technical  issues  with  the  TA  that 
represented  the  normal  glitches  in  the  development  process  of  complex  technologies. 

Utilization  and  Dissemination  of  Findings: 

Overall,  the  user  testing  revealed  four  main  findings  that  should  be  addressed  in  future 
development  cycles.  First,  the  quality  of  generic  content  will  be  critical  to  the  adoption  of  the 
tool.  Second,  training  content  that  incorporates  the  latest  information  will  be  the  most  relevant  to 
the  operational  environment,  and  that  information  is  usually  classified.  Third,  access  to 
computers  is  limited.  Fourth,  modifications  are  needed  to  course  and  activity  templates  to 
enhance  their  flexibility  and  usability.  Work  continues  to  address  the  four  issues  to  create  a  tool 
that  will  allow  operational  unit  trainers  to  easily  access  the  necessary  resources  to  create 
engaging  training  that  accurately  reflects  the  realities  of  the  current  operational  environment. 

The  results  of  this  research  have  been  communicated  to  the  U.S.  Army  Training  Support  Center, 
the  Combined  Arms  Center  -  Training,  and  to  representatives  of  the  U.S.  Army  Training 
Network. 
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Introduction 

The  rate  of  change  in  the  modem  operational  environment  requires  U.S.  Army  trainers  to 
deliver  effective  training  in  less  time  than  ever  before.  Among  the  changing  environmental 
conditions  are  enemy  tactics  (e.g.,  new  methods  for  triggering  improvised  explosive  devices), 
new  vehicles  and  equipment  (e.g.,  unattended  ground  sensors),  updated  command,  control,  and 
communications  systems  (e.g.,  Joint  Network  Node  version  updates),  unit  composition  (i.e., 
personnel  turnover),  and  even  unit  structure  (e.g.,  ad  hoc  or  interagency  transition  teams).  Each 
of  these  changes  produces  new  requirements  for  structured  training,  such  that  knowledge 
acquired  from  the  operational  environment  can  be  transfonned  into  skilled,  automatized  behavior 
(Shadrick,  Lussier,  &  Fultz,  2007).  Growth  in  training  requirements  outpaces  formal  educational 
milestones,  which  provide  structured  education  but  which  also  occur  only  a  handful  of  times  in  a 
Soldier’s  career.  For  Soldiers,  most  learning  occurs  on  the  job  and  is  specific  to  the  unit  in  which 
the  Soldier  is  operating,  so  trainers  keeping  pace  with  environmental  change  must  be  able  to 
quickly  develop  tailored  training  in  order  to  foster  human  capital  in  their  unit. 

There  are,  however,  individual  differences  in  trainers’  abilities  to  design  and  produce 
effective  training.  First,  trainers  themselves  are  often  unfamiliar  with  the  content  to  be  trained. 
This  situation  is  common  as  Soldiers  take  on  tasks  that  traditionally  have  fallen  outside  of  their 
functional  area  (Gerber,  2007)  or  tasks  that  are  new  to  the  Army  altogether  (e.g.,  U.S.  Army 
Center  for  Lessons  Learned,  2007).  Second,  many  trainers  are  unfamiliar  with  the  principles  of 
effective  instructional  design,  having  not  had  fonnal  training  in  this  area.  Indeed,  the  doctrine- 
endorsed  instructional  strategy  to  “train-as-you-fight”  (e.g.,  U.S.  Department  of  the  Anny, 
2003a),  adopted  by  most  trainers,  contrasts  with  best  practice  in  skill  development  as  identified 
by  scientific  research.  That  research  has  shown  that  experiential  learning  is  not  sufficient  for 
optimizing  skill  development,  but  that  deliberate  practice  (i.e.,  structured,  coached  exercise  on 
specific,  challenging  learning  objectives  with  performance  assessment  and  feedback;  Ericsson, 
Krampe,  &  Tesch-Roemer,  1993)  is  required  (Shadrick  et  ah,  2007). 

There  are  other  challenges  facing  trainers  who  take  the  initiative  to  create  adaptive 
training.  Chiefly,  trainers  often  are  unaware  of  the  resources  available  to  build  training  content 
(or  how  to  quickly  find  them)  because  these  resources  are  widely  distributed.  These  resources 
include  the  Center  for  Anny  Lessons  Learned  (CALL),  Battle  Command  Knowledge  System 
(BCKS)  professional  forums,  Army  Knowledge  Online  (AKO),  the  Reimer  Digital  Library 
(RDL),  the  Combined  Arms  Research  Library  (CARL),  and  unit  networks,  to  name  a  few. 
Limited  time  prevents  an  in-depth  search  of  this  information.  The  same  time  constraints  limit 
trainers’  ability  to  modify  the  information  they  find  such  that  it  is  usable  for  training  purposes. 

Therefore,  research  and  development  to  support  Soldier  training  must  explore  not  only 
advanced  learning  environments  and  instructional  strategies  but  also  advanced  training 
development  processes.  Advanced  training  development  processes  would  enable  the  rapid 
generation  of  training  activities  that  are  responsive  to  immediate  training  needs.  This  final  report 
documents  a  Phase  II  Small  Business  Technology  Transfer  (STTR)  project  to  research  and 
develop  an  integrated  platform  of  technologies  (the  Training  Assistant  [TA])  to  enable  “one- 
stop”  creation,  delivery,  and  management  of  Web-enabled  multimedia  training  in  operational 
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units  of  institutional  environments.  The  report  begins  by  summarizing  the  Phase  I  research  and 
development,  then  describes  the  Phase  II  work  in  detail.  The  report  closes  with  conclusions  and 
recommendations  for  future  research  based  on  initial  testing  and  evaluation  of  the  Phase  II 
product. 


Phase  I  Research  and  Development 

Phase  I  tasks  included  researching  the  Army  training  process,  identifying  methods  for 
relieving  the  constraints  on  rapid,  contextualized  training  development,  and  developing  these 
methods  into  a  prototype  Anny  “Training  Assistant”  capability.  The  feasibility  of  developing  a 
full-scale,  Phase  II  TA  for  use  by  unit  trainers  was  then  examined.  The  Phase  I  objectives  were 
as  follows: 

•  Determine  the  design  requirements  for  an  Internet-enabled  TA  that  supports  the  rapid 
development  of  contextualized  training,  to  include  identifying  the  (a)  mission  tasks  and 
training  objectives,  activities,  and  format  to  be  supported;  (b)  user  interaction  with  the 
tool  for  generating  structured  training;  (c)  integration  of  generic  content  and  operational 
digital  products  into  the  training  development  process;  (d)  format  of  tool  output;  and  (e) 
additional  features  to  enhance  usability  and  acceptance. 

•  Determine  the  technical  requiremen  ts  for  building  the  TA  architecture,  to  include 
identifying  the  (a)  search-and-retrieval  functions  to  access  operational  database 
resources;  and  (b)  communication  modules  necessary  to  enable  the  tool  to  communicate 
with  multiple  operational  databases. 

•  Determine  the  technical  requirements  for  developing  the  TA  front-end,  to  include  the 
order  and  links  among  Web  pages  to  structure  user  interaction  and  generation  of  output. 

•  Determine  the  feasibility  of  a  full-scale  TA  capability  based  on  the  Phase  I  research  and 
development,  to  include  identifying  (a)  challenges  to  full-scale  development;  (b)  barriers 
to  full-scale  implementation;  and  (c)  areas  of  greatest  potential  impact  of  the  full-scale 
capability  and  their  implications  for  managing  the  scope  of  Phase  II  development. 

Phase  I  Research 

The  Phase  I  effort  began  with  an  analysis  of  the  Army  training  process.  The  analysis 
consisted  of  a  review  of  relevant  doctrine  (U.S.  Department  of  the  Army,  1984,  2002a,  2002b, 
2003,  2003,  U.S.  Training  &  Doctrine  Command,  1999,  2004),  interviews  of  unit  trainers, 
observations  of  pre-deployment  training,  and  the  application  of  in-house  subject  matter  expertise. 
Interviewees  included  an  infantry  battalion  commander  recently  returned  from  deployment, 

Army  schoolhouse  instructors  located  at  Fort  Leavenworth,  KS,  and  observer/controllers  at  the 
National  Training  Center  at  Fort  Irwin,  CA.  The  purpose  of  this  analysis  was  to  gain  an 
understanding  of  the  Army’s  expectations  for  how  training  is  developed  and  conducted.  The 
results  of  this  analysis  informed  the  functional  and  technical  design  of  a  full-scale  concept  TA. 

Unit  Trainers 

The  Army  training  process  analysis  revealed  that  “unit  trainer”  can  be  conceived  broadly 
to  include  tactical  unit  commanders,  operations  staff  officers,  non-commissioned  officers, 
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schoolhouse  instructors,  and  civilian  contractors,  among  others.  Unit  trainers  may  be  thought  of 
as  being  directly  or  indirectly  involved  in  unit  readiness,  depending  on  whether  they  are  focused 
on  imparting  a  broad  education  or  on  facilitating  mission  preparation. 

Because  the  purpose  of  the  TA  is  to  support  the  rapid  development  of  contextualized 
training  activities,  the  initial  scope  of  the  TA  concept  design  was  limited  to  addressing  the  needs 
of  individuals  directly  involved  in  small-  to  moderate-scale  unit  training:  tactical  unit 
commanders,  operations  and  executive  staff  officers,  and  non-commissioned  officers.  Emphasis 
was  placed  on  mechanized  infantry  and  dismounted  armor  trainers  because  the  effective 
functioning  of  these  types  of  units  is  a  direct  determinant  of  success  in  the  contemporary 
operating  environment. 

Training  Activities 

Unit  trainers  directly  involved  in  unit  readiness  generate  a  variety  of  training  activities 
that  fall  into  one  or  more  of  three  categories  of  training:  individual,  collective,  and  leader.  The 
unit  trainer’s  selection  of  a  particular  training  activity  depends  on  who  is  being  trained  and  the 
resources  available  to  conduct  training.  Conducting  training  in  this  way  is  consistent  with  the 
Army’s  crawl,  walk,  run  progression  of  training  difficulty,  with  the  expectation  that  trainees 
would  be  challenged  at  a  level  that  optimizes  the  trade-off  between  resource  expenditure  and 
training  benefit. 

The  TA  concept  was  designed  to  enable  the  rapid  development  of  training  via  two  main 
functionalities.  First,  the  TA  would  allow  unit  trainers  to  create  new  training  from  scratch, 
assembling  novel  and  existing  materials  to  create  a  variety  of  training  activities.  This 
functionality  is  the  heart  of  the  TA  and  its  core  capability.  Second,  the  TA  would  allow  unit 
trainers  to  modify  existing  training  that  has  been  developed  by  others  who  have  used  the  TA  to 
create  their  own  new  training.  This  second  functionality  was  intended  to  help  trainers  avoid 
duplication  of  effort  by  making  the  efforts  of  others  easy  to  access  and  build  upon  (Kilner,  2002). 
Figure  1  below  depicts  the  training  activities  supported  in  the  TA  concept.  This  selection  of 
training  activities  would  enable  the  development  of  individual,  collective,  and  leader  training 
activities  that  are  not  the  primary  responsibility  of  institutional  education.  The  TA  concept  was 
designed  to  place  emphasis  on  collective  and  leader  training  activities,  which  are  difficult  to 
create  but  critically  important  to  mission  success. 
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Figure  1 .  Training  Activities  Supported  in  the  TA  Concept.  The  activities  presented  would 
enable  the  development  of  training  that  is  not  the  primary  responsibility  of  institutional 
education. 

Note:  See  Appendix  A  for  acronym  list. 

The  Conduct  of  Training 

Training  may  be  conducted  in  a  variety  of  ways,  with  differing  implications  for  what  unit 
trainers  must  develop  to  conduct  a  particular  training  activity.  Individual  training  often  is 
administered  in  the  form  of  a  computer-based  training  package,  which  can  be  delivered  via  the 
Web  or  on  a  compact  disc.  Unit  trainers  therefore  develop  individual  training  less  often  than  they 
must  monitor  such  training  to  ensure  that  Soldiers  and  leaders  meet  the  training  requirements. 
Exceptions  include  the  seminar/lecture  and  training  on  individual  practical  skills.  Collective 
training  may  be  conducted  as  live  exercises,  computer-simulated  exercises,  tabletop  exercises 
(technology-enabled  or  otherwise),  or  as  simple  thought  exercises.  In  any  case,  the  unit  trainer 
must  develop  background  materials  to  set  the  conditions  for  the  exercise  and  to  situate  the 
trainees.  Leader  training  may  fall  under  the  rubric  of  individual  or  collective  training,  depending 
on  the  training  activity. 

The  TA  concept  was  designed  to  produce  a  different  output,  depending  on  the  training 
activity  and  format  selected.  Output  could  range  from  printable  background  materials  to 
seminar/lecture  presentation  materials,  to  interactive  computer-based  exercises.  For  easy  access, 
all  output  would  be  in  the  form  of  Web  pages.  These  could  be  printed,  presented,  shared,  and 
viewed  simultaneously  (though  not  in  a  distributed/collaborative  fashion)  without  specialized 
software  available  on  the  computer  by  which  the  training  is  accessed.  The  Internet-enabled 
format  of  the  output  would  allow  trainers  and  trainees  alike  to  access  the  TA  from  any  computer 
that  has  Internet  access,  whether  deployed  or  stateside. 

Doctrinal  Support  for  Training 

Unit  trainers  generally  attempt  to  make  their  training  as  doctrinally  sound  as  possible. 

The  Army  supports  their  efforts  by  providing  numerous  resources  for  planning,  conducting,  and 
assessing  training  activities.  The  length  and  quantity  of  doctrinal  training  manuals  slows  down 
the  training  development  process  because  substantial  time  is  required  to  locate  appropriate 
manuals  and  implement  their  guidance  in  training  exercises.  Moreover,  many  training  developers 
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may  be  unaware  that  doctrinal  support  exists  and  therefore  do  not  seek  it  out.  Currently,  there  is 
no  simple  way  for  unit  trainers  to  locate  segments  of  doctrine  for  relevant  training  guidance, 
although  searching  for  entire  manuals  is  relatively  straightforward  via  portals  such  as  Army 
Knowledge  Online  (AKO).  Many  doctrinal  manuals,  including  training  publications,  are  well 
over  200  pages  long,  requiring  non-trivial  download  times  (especially  for  trainers  using  dial-up 
Internet  connections)  and  significant  review  time  to  locate  topics  of  interest. 

These  findings  indicate  that  the  TA  could  significantly  enhance  the  training  development 
process  by  making  doctrinal  resources  readily  available  via  a  centralized  source  and  in  a  format 
that  enables  easy  insertion  into  training  activities.  Figure  2  features  the  high-level  architecture  of 
the  TA  concept  system.  As  shown  in  the  figure,  the  selection  of  a  training  activity  by  the  unit 
trainer  determines  the  set  of  queries  the  TA  presents  to  support  the  development  of  training 
materials  for  that  activity.  Through  this  interaction  with  the  TA,  the  trainer  provides  training 
content  to  the  system  for  assembly  into  a  complete  set  of  training  materials,  including  the  content 
of  the  training  activity  itself  as  well  as  perfonnance  assessment  criteria  and  feedback  or  after 
action  review  (AAR)  materials,  where  applicable.  The  TA  concept  also  provides  content  for  the 
trainer.  This  content  is  based  directly  or  indirectly  on  doctrinal  resources  and  is  relatively  generic 
and  modifiable  to  ease  the  training  development  process. 


Training  Activity 


Trainee  Type 


t 


Platoon 

Company 

Battalion 

Leader 

Commander 

Staff 

Figure  2.  High-Level  Architecture  of  TA  Concept  System.  The  selection  of  a  specific  training 
activity  determines  the  set  of  queries  that  the  TA  will  present  to  support  the  development  of 
training  materials  for  that  activity. 

Operational  Databases  and  Training 

Unit  trainers  contextualize  training  by  matching  the  conditions  and  (to  some  extent)  the 
standards  used  in  a  training  activity  to  those  conditions  and  performance  requirements  expected 
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to  characterize  a  particular  operational  setting.  There  are  two  ways  in  which  resources  from 
operational  databases  can  be  used  to  provide  context  for  training.  First,  elements  in  a  database 
can  be  embedded  directly  into  the  training  activities  themselves.  Training  developers  at  the  Fort 
Lewis  (WA)  Battle  Command  Training  Center  use  this  approach  to  integrate  theater  conditions 
and  lessons  learned  directly  into  the  training  for  units  preparing  to  deploy  (Jean,  2005). 

Second,  multiple  elements  in  an  operational  database,  such  as  informal  documents  and 
text  messages,  can  be  reviewed  and  synthesized  to  inform  the  development  of  training  conditions 
and  standards  for  mission  tasks  that  are  not  well  established  in  doctrine  (i.e.,  new  techniques  and 
procedures).  Synthesized  operational  database  material  may  also  be  used  to  customize  the 
conditions  for  established  mission  tasks,  but  in  a  more  indirect  way.  Training  developers  at  the 
National  Training  Center  adopt  this  second  approach.  They  use  lessons  learned  from  theater  to 
determine  opposing  force  tactics,  civilian  behaviors,  and  urban  terrain  characteristics  that  form 
the  learning  environment. 

In  order  to  support  the  development  of  contextualized  training,  therefore,  the  TA  must 
enable  users  to  embed  operational  products  within  a  training  activity  and  must  provide  easy 
access  to  the  other  materials  available  in  the  operational  database  for  review  and  synthesis.  The 
TA  concept  design  assumes  a  single  operational  database,  in  effect  a  tagged  fde  repository, 
consistent  with  the  vision  that  at  some  future  time  all  Army  operational  databases  will  be  unified 
or  interoperable.  Because  a  single  operational  database  does  not  yet  exist,  the  TA  concept 
capability  was  designed  to  communicate  flexibly  with  different  databases  so  that  it  can  be 
implemented  easily  in  a  variety  of  locations.  Figure  2  shows  where  in  the  training  development 
process  the  TA  concept  integrates  operational  products.  As  TA  queries  request  content  from  the 
user,  the  user  may  provide  content  by  selecting  digital  products  that  are  then  embedded  into  the 
output  of  the  TA. 


Phase  I  Development 

In  Phase  I,  a  prototype  TA  was  developed  to  conduct  a  feasibility  assessment  of  the  TA 
concept.  The  Phase  I  prototype  design  featured  a  selection  of  the  full-scale  TA  concept 
capabilities  that  best  enabled  the  assessment. 

Of  the  TA  concept  functions  previously  described,  we  focused  Phase  I  effort  on  authoring 
new  training  activities  from  scratch  because  this  functionality  is  the  heart  of  the  TA  and  its  core 
capability.  To  manage  the  scope  of  the  Phase  I  effort  we  designed  the  tool  to  support  company 
commanders  as  trainers  and  platoon  leaders  as  trainees.  These  leaders  are  likely  to  have  the  most 
rapidly  changing  training  objectives  and  the  least  experience  developing  structured  training.  In 
addition,  we  focused  our  attention  on  developing  the  capability  to  author  a  decision-making 
exercise  (DMX). 

We  used  the  Think  Like  a  Commander  (TLAC)  adaptive-thinking  training  methodology 
(Lussier,  Shadrick,  &  Prevou,  2003)  as  the  basis  for  designing  the  output  of  the  prototype  TA 
DMX  authoring  capability.  That  is,  the  DMX  output  is  designed  to  present  to  trainees  a  brief 
scenario  depicting  an  ongoing  situation  (e.g.,  a  patrol  operation),  its  associated  context  (e.g., 
higher’s  mission  and  intent,  status  of  friendly  forces,  general  enemy  tactics,  etc.),  and  a  set  of 
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unexpected  events  that  occur  during  the  ongoing  situation.  The  DMX  output  requires  trainees  to 
indicate  the  factors  they  would  consider  in  deciding  how  to  handle  the  unexpected  events  and 
allows  trainees  to  compare  the  factors  they  identified  relative  to  those  identified  by  an  expert  (or 
experts). 

The  unit  trainer  creates  a  training  activity  through  a  series  of  queries  that  are  specific  to 
that  training  activity.  To  create  a  DMX,  the  queries  shown  in  Table  1  below  are  used  and 
information  provided  by  the  user  (queries  listed  in  order,  grayed  out  areas  are  optional).  A 
subject  matter  expert  developed  the  generic  content  for  the  prototype  TA,  which  is  present  in  all 
of  the  required  fields  listed  in  Table  1  (except  the  selection  of  the  background  task).  In  general, 
fields  in  the  generic  content  database  are  called  up  depending  on  the  background  task  and  trainee 
type  selected  by  the  unit  trainer.  However,  some  fields  (e.g.,  rules  of  engagement,  civilians, 
timing)  are  generic  across  all  trainee  types  and  background  tasks.  Generic  content  appears  to  the 
user  as  text  already  filled  into  a  field,  which  may  be  modified  or  deleted,  depending  on  the  user’s 
preference. 

Table  1 

DMX  User  Queries 


Query 

Form  of  User  Input 

Select  Background  Task 

Pull-down  menu  selection  or 
text  entry 

Select  the  Area  of  Operations  (AO) 

File  upload 

Specify  Key  Terrain  Features 

Text  entry 

Specify  Higher  Mission  &  Intent 

Text  entry 

Specify  Higher  Task  Organization 

Text  entry 

Specify  ROE/ROI 

Text  entry  or  file  upload 

Indicate  Additional  Context 

Text  entry  or  file  upload 

Specify  Enemy  General  Tactics 

Text  entry 

Specify  Additional  Enemy  Intelligence 

Text  entry  or  file  upload 

Specify  Recent  Enemy  Activity  in  the  AO 

Text  entry  or  file  upload 

Specify  Task  Organization  -  Own  Unit 

Text  entry 

Specify  Personnel/Equipment  Status 

Text  entry 

Specify  Other  Military  Elements  in  the  AO 

Text  entry 

Specify  Timing 

Text  entry 

Specify  Civilians 

Text  entry 

Specify  Unexpected  events 

Text  entry 

Specify  Factors  to  consider  -  Keep  a  Focus  on 

Mission/Higher’s  Intent 

Text  entry 

Specify  Factors  to  consider  -  Model  a  Thinking  Enemy 

Text  entry 

Specify  Factors  to  consider  -  Consider  the  Effects  of  Terrain 

Text  entry 

Specify  Factors  to  consider  -  Use  all  Assets  Available 

Text  entry 

Specify  Factors  to  consider  -  Consider  Timing 

Text  entry 

Specify  Factors  to  consider  -  See  the  Bigger  Picture 

Text  entry 

Specify  Factors  to  consider  -  Visualize  the  Battlefield 

Text  entry 

Specify  Factors  to  consider  -  Consider  Contingencies  and 

Remain  Flexible 

Text  entry 

Note:  Shaded  rows  are  optional  fields. 
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There  are  several  opportunities  for  the  unit  trainer  to  embed  operational  database 
resources  into  the  DMX.  These  opportunities  include  inserting  (a)  depictions  of  a  deployed  unit’s 
area  of  operations  (i.e.,  map  fdes  or  map  fdes  with  graphics,  photographs,  sketches,  or  other 
graphic  fdes);  (b)  rules  of  engagement/interaction  used  in  theater;  (c)  additional  contextualizing 
materials  from  an  area  of  operations  in  theater  (e.g.,  photographs,  weather  reports,  incident 
reports,  etc.);  (d)  additional  enemy  intelligence  from  deployed  units’  reports  (i.e.,  intelligence 
reports  such  as  person  files,  pattern  analyses,  etc.);  and  (e)  infonnation  on  recent  enemy  activity 
(i.e.,  fragmentary  orders  [FRAGO]). 


Phase  I  Findings 

Subject  matter  experts  suggested  that  the  TA  concept  has  great  potential  for  saving  time, 
increasing  productivity,  and  improving  training.  A  full-scale  capability  enabling  the  development 
of  several  collective  exercises  and  individual,  small-group  training  activities  would  have  broad 
impact  and  would  be  doctrinally  sound  and  easy  to  use.  Even  after  effort  to  manage  the  scope  of 
the  TA  concept,  however,  it  was  determined  that  implementing  the  full-scale  concept  capability 
could  not  feasibly  be  accomplished  in  a  Phase  II  effort. 

Rather,  it  was  found  that  the  most  feasible,  influential,  and  immediately  usable  Phase  II 
implementation  of  the  TA  concept  should  focus  on  supporting  junior  officers  in  the  development 
of  training  activities  not  supported  currently  by  doctrine,  especially  the  DMX.  The  DMX  is 
intended  to  train  flexible  responses  to  rapidly  changing  enemy  tactics  or  other  unexpected  events, 
a  training  objective  not  well  addressed  by  other  doctrinal  training  activities  but  of  greatest 
importance  to  junior  officers.  Because  the  DMX  does  not  have  a  doctrinal  structure,  it  is  more 
subject  to  the  lack  of  training  development  experience  most  junior  officers  have.  Providing  a 
structured  DMX-development  process  based  on  best  practice  would  enhance  the  training  quality 
of  DMXs  Army- wide  through  reduced  individual  differences  in  instructional-design  savvy  and 
increased  average  capability.  The  DMX  is  also  relatively  easy  to  generate  and  simple  to  conduct 
in  the  short  timeframe  available  to  train  rapidly  changing  techniques  and  procedures. 

It  was  determined  that  there  are  several  different,  feasible  ways  in  which  the  Phase  I 
prototype  TA  could  be  scaled  up  to  an  operational  Phase  II  product  such  that  it  enabled  the  rapid 
development  of  contextualized  DMXs  or  other  vignette -based  individual,  small-group  exercises. 
Effective  scaling  up  must  enhance  (a)  the  basic  functionality  of  the  tool  to  support  modification 
of  developed  training;  (b)  the  range  of  methods  the  unit  trainer  can  use  to  create  training 
exercises;  (c)  the  instructional  and  technological  sophistication  of  the  TA  output;  and  (d)  the  ease 
with  which  users  can  access  resources  from  the  operational  environment. 

The  Phase  I  analysis  of  the  Army  training  process  directly  infonned  the  Phase  II  effort  by 
identifying  the  common  mission  tasks  most  likely  to  be  affected  by  rapidly  changing  conditions 
in  the  operational  environment,  providing  a  characterization  of  target  audience  expectations  for 
the  tool  interface  and  output,  and  specifying  the  use  of  generic  content  and  operational  digital 
products  in  contextualizing  a  DMX.  The  Phase  I  technical  analysis  informed  the  Phase  II  effort 
by  determining  the  search-and-retrieval  techniques  necessary  to  link  operational  database  content 
to  relevant  components  of  the  training  development  process. 
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Phase  II  Objectives 


The  operational  Phase  II  product  was  intended  to  support  unit  trainers  in  rapidly  creating 
DMXs  involving  digital  products  from  theater  in  response  to  changes  in  the  operational 
environment.  Specifically,  the  Phase  II  product  was  intended  to  enable  the  unit  trainer  to  rapidly 
develop  interactive  multimedia  instruction  that  could  support  group  discussion  and  feedback  and 
that  could  train  squad-,  platoon-,  and  company-level  Soldiers  and  leaders  to  identify  the 
decision-making  and  mission-performance  implications  of  specific  changes  in  the  operational 
environment,  to  include  (but  not  be  limited  to)  changes  in: 

•  Enemy  tactics; 

•  Rules  of  engagement/rules  of  interaction; 

•  Communication  standing  operating  procedures  (SOP); 

•  Host  nation  civilian  attitudes/behaviors; 

•  Armor/equipment  usage  principles/procedures; 

•  Media  standing  operating  procedures;  and 

•  Task  organization. 

The  objectives  for  the  Phase  II  effort  involved  expanding  significantly  on  the  capabilities 
of  the  Phase  I  prototype  including: 

1 .  The  development  of  interactive  multimedia  instruction  using  a  combination  of  audio, 
text,  and  graphics,  plus  integration  of  digital  operational  products; 

2.  Structured  user  queries  that  are  more  broadly  applicable  than  those  required  for 
conducting  Think  Like  a  Commander  adaptive-thinking  training  (Lussier  et  ah,  2003); 
and, 

3.  An  enhanced  working  operational  database  and  search  capability  and  multimedia  generic 
content  to  enable  speed  in  generating  relevant  exercises. 

The  objectives  for  the  Phase  II  research  therefore  were  as  follows: 

1 .  Determine  the  architecture  required  for  the  TA  to  enable  the  development  and  use  of 
interactive  multimedia  instruction  that  addresses  rapidly  changing  conditions  in  the 
operational  environment; 

2.  Determine  the  algorithms  necessary  to  develop  an  Internet  crawler  that  enables  the 
population  of  a  self-updating,  adaptable  working  operational  database  using  products 
from  unit  databases; 

3.  Determine  the  algorithms  necessary  to  limit  the  searchable  space  of  the  working 
operational  database  based  on  individual  user  preferences; 

4.  Develop  the  architecture  and  crawler/search  algorithms,  implement  these  components  in 
a  coherent  whole  system;  and 

5.  Test  and  evaluate  the  system  functionality. 
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Phase  II  Research 


Behavioral  Research  and  TA  Design 

This  section  describes  the  behavioral  research  conducted  to  refine  the  TA  design  based 
on  the  Phase  I  research  findings.  This  research  included  further  examination  of  target  user 
characteristics  and  the  likely  use  environment.  The  purpose  of  this  research  was  to  ensure  that 
the  TA  would  optimally  support  the  flexible  development  of  DMXs  and  related  training  in  a  way 
that  is  easy  to  use  and  integrated  into  a  larger  instructional  context  that  individualizes  training. 
The  research  findings  and  design  specifications  presented  in  this  section  represent  the  ideal  case, 
a  completed  system  that  represents  a  Phase  III  capability.  Decisions  to  manage  the  scope  of  the 
Phase  II  product  were  made  and  are  described  in  the  Phase  II  Development  section. 

Method 

Phase  II  behavioral  research  was  conducted  via  a  literature  review  and  a  series  of 
interviews,  both  archived  (i.e.,  conducted  as  part  of  another  effort)  and  conducted  as  part  of  the 
present  effort.  The  literature  review  included  the  scholarly  and  technical  literature  regarding 
interactive  multimedia  instruction  design,  evaluation,  and  authoring  (e.g.,  Hays,  Stout,  &  Ryan- 
Jones,  2005;  Herrington  &  Oliver,  1995;  Lee  &  Boling,  1999;  Park  &  Hannafin,  1993).  The 
intent  of  this  literature  review  is  to  capture  the  human  factors  issues  that  must  be  taken  into 
consideration  when  designing  the  TA,  including  interface  features,  training  performance 
assessment,  and  promotion  of  trainee  engagement  in  higher-order  thinking. 

The  series  of  interviews  was  as  follows:  (1)14  archived  interviews  with  veterans  of 
Operation  Iraqi  Freedom  (OIF)  [located  in  the  digital  library  of  the  U.S.  Army  Command  and 
General  Staff  College  Combined  Arms  Research  Library  (CARL)];  (2)  two  focus  groups 
conducted  with  20  Soldiers  of  the  7th  Squadron,  First  Cavalry  Division  immediately  prior  to  their 
deployment  to  Iraq  in  October  2006;  and  (c)  interviews  with  10  observer/controllers  at  the  Joint 
Readiness  Training  Center  (JRTC)  at  Fort  Polk,  LA.  The  majority  of  the  interviewees  and  focus 
group  members  (33/44)  were  enlisted  Soldiers  ranging  from  specialist  to  command  sergeant 
major.  The  remaining  interviews  (11/44)  were  conducted  with  platoon  leaders  (lieutenants)  and 
company  commanders  (captains).  Interviewees  and  focus  group  members  were  annor,  cavalry, 
infantry,  military  police,  and  field  artillery  Soldiers. 

Findings 

User  Characteristics  and  Preferences  -  Trainers.  Trainers  comprise  battalion 
commanders/S3s,  company  commanders  (CO  CDRs),  platoon  leaders  (PLs),  and  non¬ 
commissioned  officers  (NCOs)  or  other  enlisted  personnel  (especially  first  sergeants  and  platoon 
sergeants,  but  also  possibly  squad  and  team  leaders)  in  the  active  or  reserve  component,  who 
lead  full-spectrum  operations.  Unit  trainers  also  may  include  Department  of  the  Army  (DA) 
civilians  or  contractors  located  at  a  mission  support  training  facility  (MSTF)  or  home  station 
operations  center  (HSOC).  Trainers  also  include  schoolhouse  instructors  responsible  for 
developing  junior  officers  and  enlisted  Soldiers  (e.g.,  NCO  Academy  or  Platoon  Leaders’ 
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Development  Course  instructors).  Schoolhouse  instructors  may  be  active  or  reserve  Army 
personnel,  DA  civilians,  or  contractors. 

Trainers  would  use  the  TA  to  rapidly  develop  foundational  interactive  multimedia 
exercises  for  individuals  and/or  small  groups.  They  also  would  access  the  TA  to  use/adapt 
exercises  developed  by  other  trainers  and  located  in  the  TA  archives.  Foundational  exercises 
should  address  single  competencies  or  behaviors  (e.g.,  react  to  ambush,  coordinate 
mounted/dismounted  operations,  etc.)  in  order  to  enable  deliberate  practice  and  skill  adaptability 
across  a  range  of  conditions  (Lussier  &  Shadrick,  2004).  Trainers  also  would  use  the  TA  to 
develop  various  types  of  small  unit  or  key  leader  mission  rehearsals  (i.e.,  “digital  tabletop,” 
briefback,  confirmation  brief,  and  pre-combat  inspection/check). 

Unit  trainers  may  approach  the  creation  of  exercises  with  a  particular  doctrinal  task  they 
wish  to  train  (e.g.,  report  tactical  information).  Alternatively,  unit  trainers  may  approach  training 
with  a  general  competency  they  wish  to  develop  (e.g.,  communications  skills).  This  competency 
generalizes  across  task  types,  although  trainers  may  wish  to  situate  general  competency  training 
within  a  particular  task  context  (e.g.,  communications  skills  during  a  raid).  Schoolhouse 
instructors  likely  will  approach  instruction  from  an  educational  (as  opposed  to  training) 
standpoint,  wishing  to  develop  general  competencies,  such  as  adaptive  thinking,  military 
decision-making,  etc.  To  illustrate,  unit  trainers  may  seek  to  train  “react  to  contact  [specifically 
improvised  explosive  devices  (IEDs)  on  a  particular  convoy  route  to  be  secured]”  whereas 
schoolhouse  instructors  may  seek  to  expand  Soldiers’  and  leaders’  mental  models  of  what  a 
thinking  enemy  can  do  to  reduce  the  likelihood  of  friendly  success. 

Trainers  are  familiar  with  mainstream  software  applications  (especially  Microsoft®  Word 
and  PowerPoint),  Internet  browsers  (especially  Microsoft®  Internet  Explorer),  search  engines 
(especially  Google),  knowledge  management  portals  (especially  Wikipedia,  Questia,  and  Army- 
specific  portals),  and  computer-based  instruction.  Commissioned  officers,  DA  civilians,  and 
contractors  may  have  more  familiarity  with  some  software  applications  than  NCOs  and  junior 
enlisted  Soldiers,  given  (a)  the  greater  demand  on  these  personnel  to  prepare  briefings  and  other 
documents;  (b)  their  generally  higher  levels  of  education;  and  (c)  their  stronger  orientation 
toward  an  “office”  setting.  However,  NCOs  and  junior  enlisted  Soldiers  in  the  reserve 
component  may  have  more  experience  than  their  counterparts  in  the  active  component,  due  to 
their  civilian  jobs  and  education  and  their  participation  in  Army  distributed  learning.  Nearly  all 
trainers  are  familiar  with  other  mainstream  technologies,  including  cell  phones,  digital  cameras, 
and  personal  digital  assistants.  A  small  group  of  target  users  are  advanced  users  of  these 
technologies,  using  them  innovatively  in  the  field  and  for  personal  creative  purposes. 

With  the  possible  exception  of  some  schoolhouse  instructors  and  MSTF/HSOC 
personnel,  few  trainers  have  fonnal  education  in  instructional  design.  In  designing  training,  most 
trainers  prioritize  realism  over  all  other  instructional  characteristics.  Where  high-fidelity 
simulation  can  be  achieved,  even  at  the  cost  of  other  instructional  best  practices  (including 
formal  assessment  and  feedback),  it  usually  is  preferred  (see  Lussier  &  Shadrick,  2004).  Aside 
from  the  mandate  of  training  doctrine  (DA,  2003),  one  reason  for  the  preference  of  realism  may 
be  the  possibility  that  high-fidelity  simulation  is  better  for  developing  the  psychomotor, 
perceptual  speed,  and  stress-coping  skills  involved  in  close-quarters  combat.  Schoolhouse 
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instructors  may  place  lower  priority  on  realism,  given  the  emphasis  their  proponent  schoolhouse 
plays  on  general  education  over  mission  preparation.  In  any  case,  the  TA  must  have  best  practice 
in  instructional  design  infused  into  its  authoring  component  so  that  the  training  presented  to 
trainees  conforms  to  widely  recognized  guidelines  for  effective  instruction. 

The  design  of  the  TA  must  resonate  with  trainers’  understanding  of  doctrine  but  be 
flexible  enough  to  address  necessary  deviations  from  doctrinal  practices,  such  as  “train-as-you- 
fight.”  The  rationale  for  and  utility  of  using  simple,  targeted  computer-based  exercises  for 
individuals  and  small  groups  must  be  made  clear  to  trainers.  For  instance,  the  TA  will  prepare 
units  for  larger-scale  collective  exercises  and  it  is  a  feasible  training  solution  if  collective 
exercises  are  not  feasible.  In  addition,  the  TA  is  especially  useful  for  key  leader  development. 

The  argument  that  targeted  computer-based  exercises  are  better  education  than  live  action 
collective  exercises  likely  will  not  resonate  with  users  and  is  not  recommended  for  generating 
interest  in  and  engagement  with  the  TA. 

User  Characteristics  and  Preferences  -  Trainees.  Trainees  comprise  junior  enlisted 
Soldiers,  NCOs,  and  junior  commissioned  officers  learning  in  line  units  or  at  the  schoolhouse. 
Trainees  may  be  in  the  active  or  reserve  component.  With  regard  to  familiarity  with  computers, 
software  applications,  and  other  technology,  trainees  have  similar  characteristics  as  trainers. 
Trainees  would  use  the  TA  to  access  and  conduct  training  developed  by  their  unit  trainers  or 
instructors.  Trainees  also  may  access  the  TA  to  conduct  self-development  using  training  residing 
in  the  TA  archives. 

The  exercises  produced  by  the  TA  must  be  easy  for  trainees  to  search,  locate,  and  access. 
The  exercises  must  be  engaging  and  performance  feedback  must  be  immediate  and  perceived  as 
fair.  Some  non-instructional  bells  and  whistles,  such  as  background  music  and  two-dimensional 
animation,  should  be  judiciously  considered  to  enhance  motivation  to  use  the  tool  without 
making  large  demands  on  bandwidth  or  becoming  too  distracting. 

When  the  TA  Will  Be  Used.  Trainers  will  use  the  TA  to  develop,  use,  or  adapt  exercises  in 
response  to  a  recognized  training/educational  need.  Triggers  for  this  need  will  include  (1) 
changes  in  the  operational  environment;  (2)  receipt  of  a  mission;  (3)  unit  or  student  performance 
deficits  (e.g.,  as  revealed  by  an  after-action  review  or  student  exercise);  (4)  gaps  between  unit 
skill  sets  and  mission  requirements;  (5)  the  need  to  illustrate  an  instructional  point  (as  in 
classroom  instruction);  and  (6)  recognition  that  training  exercises  will  boost  morale  and  unit 
cohesion  in  a  relatively  low-intensity  operational  environment. 

Several  types  of  changes  in  the  operational  environment  may  trigger  a  training 
requirement.  Those  most  frequently  occurring  changes  include: 

•  Changes  in  enemy  tactics  due  to  the  adaptability  of  a  thinking  enemy  or  changes  in  the 
enemy  itself  [e.g.,  changes  in  IED  employment  methods  (including  location,  time  of  day, 
delivery  and/or  triggering  mechanism,  strength,  etc.),  changes  in  IED  technology]; 

•  Changes  to  task  type  due  to  a  greater  demand  for  certain  skill  sets  than  are  present  in  a 
particular  task  organization,  requiring  the  development  of  established  skill  sets  in  novel 
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functional  areas  (e.g.,  infantry  Soldiers  doing  military  police  or  medic  tasks,  field  artillery 
or  engineers  doing  infantry  tasks,  etc.); 

•  Changes  in  mission  type  due  to  adaptive,  last-minute  changes  given  by  higher,  last- 
minute  notification  of  mission,  and  sudden  changes  in  the  operational  environment  (e.g., 
enemy  contact  in  the  context  of  security  or  reconstruction  missions  or  public  affairs 
reaction  in  the  context  of  combat  missions,  etc.); 

•  Changes  in  the  equipment  used  to  perfonn  mission  tasks  due  to  the  receipt  of  new 
equipment  in  the  field; 

•  Changes  in  personnel  in  the  unit  due  to  adaptive,  last-minute  task  organizations  or 
personnel  arrangements,  the  execution  of  combat  missions  while  some  personnel  are  on 
leave,  or  casualties;  and 

•  Changes  in  coordination  requirements  due  to  joint  operations  with  host  nation  security  or 
military  forces,  with  sister  service  units  (e.g.,  Marines),  and/or  with  allied  forces. 

Characteristics  of  the  TA  Use  Environment.  All  target  users  (trainers  and  trainees)  will 
require  a  system  that  is  extremely  easy  to  use  and  that  offers  some  unique  capability.  Trainers 
have  several  easily  accessible  and,  in  some  cases,  higher  fidelity  alternatives  with  which  they  are 
already  familiar.  These  alternatives  include  tabletop  exercises  using  paper  maps  with  acetate 
overlays,  photographs,  or  scale  models,  Microsoft®  PowerPoint  presentations,  group  discussions 
with  whiteboards  or  other  shared  visual/marking  space,  immersive  tactical  and  cultural  trainers, 
and  equipment  simulators.  The  TA  must  require  little  or  no  training  to  use,  must  be  consistent 
(ideally,  integrated)  with  the  technologies  that  users  are  already  familiar  with,  and  must  be 
stable. 


The  TA  should  be  accessible  anytime,  whenever  training/education  is  required  —  during 
pre-deployment,  the  reception,  staging,  onward  movement,  and  integration  (RSOI)  phase  of 
deployment  (e.g.,  in  Kuwait),  deployment  itself,  and  formal  education.  Pre-deployment  training 
should  comprise  exercises  to  prepare  units  for  rotations  through  combat  training  centers  and  to 
address  perfonnance  deficits  revealed  by  such  rotations.  Pre-deployment  exercises  using 
classified  operational  products  (actual  or  “scrubbed”)  from  the  unit  to  be  replaced  would  enable 
training  exercises  that  simulate  the  specific  environment  to  which  a  unit  will  be  deployed  (Jean, 
2005).  Pre-deployment  exercises  could  involve  general  skill  development  when  deployment  is 
anticipated  but  specific  orders  have  not  yet  been  delivered.  The  TA  could  be  used  during 
deployment  by  trainers  conducting  refresher  or  sustainment  training,  new  skills  training,  leader 
development,  and  mission  rehearsals.  Schoolhouse  instructors  could  use  the  TA  at  any  time,  for 
example,  following  a  class  period,  to  create  learning  experiences  that  are  adaptive  to  student 
need  or  that  illustrate  an  instructional  point.  Initially,  the  most  common  use  of  the  TA  will  be  for 
pre-deployment  training  and  formal  education  because  access  to  the  Internet  (both  classified  and 
unclassified)  is  limited  in  theater. 

Figure  3  below  shows  the  access  that  different  types  of  trainers  have  to  the  public  and 
classified  internets,  depending  on  location.  Dashed  lines  indicate  that  access  is  variable, 
depending  on  such  factors  as  the  sophistication  of  Army  presence  in  theater  or  the  possession  of 
a  common  access  card  (CAC)  reader.  Note  in  this  figure  that  reserve  component  (RC)  trainers  do 
not  have  access  to  classified  training  locations  on  post  or  at  their  schoolhouse  locations,  located 
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at  reserve  facilities,  whereas  active  component  (AC)  trainers  do.  Trainee  access  to  the  public  and 
classified  internets,  regardless  of  installation  type,  is  assumed  to  mirror  trainer  access. 


Unit  Trainer 


Schoolhouse  Instructor 


Installation 


Classified 


Theater 


Classroom 
on  Post 

(including  TASS  BNs) 


Virtual 
Classroom 
(home  computer,  DTF) 


Public 
Internet 
(and  restricted 
Army  websites) 


Classified 
Training 
Location 
(e.g.,  CARL) 


Classified 

Internet 


Internet 
(and  restricted 
Army  websites) 


Classified 

Internet 


Figure  3.  Access  to  Public  Versus  Classified  Internet,  by  Trainer  and  Location  Type.  Dashed 
lines  indicate  variable  access. 

Note:  See  Appendix  A  for  Acronyms  List. 

Target  Skills/Competencies  Trained  by  the  TA.  The  TA  should  help  trainers  address  a 
subset  of  the  training  needs  most  frequently  identified  by  deployed  or  recently  deployed  leaders 
and  Soldiers  engaged  in  full-spectrum  operations.  Although  the  TA  would  be  useful  for 
enhancing  particular  skills  in  specific  contexts  (e.g.,  identify  the  location  of  enemy  snipers 
during  a  dismounted  patrol)  or  coordinating  collective  activity  (e.g.,  as  in  mission  rehearsal  or 
synchronization  training)  the  ideal  use  of  the  TA  is  for  developing  adaptive  responses  to  sudden 
changes  in  the  operational  environment  during  the  execution  of  a  variety  of  mission  tasks, 
including  high-intensity  conflict.  The  TA  would  be  useful  for  developing  adaptive  responding 
because  its  design  enables  trainers  to  create  exercises  that  require  the  same  skill  sets  under  a 
variety  of  conditions.  That  use  of  the  TA  would  leverage  its  technological  capabilities  to 
accomplish  learning  outcomes  that  are  more  difficult  or  impossible  to  achieve  using  other  means 
(i.e.,  because  simulating  multiple  conditions  quickly  is  not  feasible). 

The  TA  should  not  be  used  to  address  the  subset  of  common  training  needs  that  are  either 
better  trained  using  accessible,  higher-fidelity  simulation  (e.g.,  small  arms  marksmanship)  or  that 
are  already  addressed  via  interactive  multimedia  instruction  used  in  Army  distributed  learning 
(e.g.,  equipment  simulations  available  through  LandWarNet  e-Signal).  The  top-level  categories 
of  skills  that  should  addressed  by  the  TA  include  the  following: 

•  Adaptive  thinking/contingency  planning; 

•  Application  of  rules  of  engagement  and  rules  of  interaction; 

•  Communications  skills; 

•  Information  management; 
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•  Interpersonal/cultural  skills; 

•  Medical  skills; 

•  Procedural  skills; 

•  Rapid  reaction  to  enemy  contact; 

•  Synchronization  and  coordination; 

•  Tactical  decision-making; 

•  Tactical  perception; 

•  Various  mission  tasks  (as  in  mission  preparation  or  rehearsal);  and 

•  Visual  communications  skills. 


The  nature  of  the  skills  or  competencies  to  be  developed  is  somewhat  dependent  on 
where  the  training  takes  place,  because  training  location  has  implications  for  access  to  the  public 
versus  classified  internet  and  for  the  relative  emphasis  placed  on  specific  skill  acquisition  versus 
general  competency  development.  For  example,  general  competency  development  is  more  likely 
to  occur  in  the  schoolhouse  setting  whereas  mission  rehearsal  is  not  expected  to  occur  in  the 
schoolhouse  at  all.  In  addition,  in  the  schoolhouse  setting,  the  likelihood  of  using  the  classified 
internet  for  training  of  any  type  is  reduced  due  to  the  relatively  minor  educational  payoff 
achieved  with  great  difficulty  accessing  and  “scrubbing”  classified  products  or  conducting 
classified  training.  Figure  4  below  shows  the  expected  relation  between  training  location  and 
type  of  training,  with  dashed  lines  indicating  reduced  likelihood  of  training  type  and  absent  lines 
indicating  zero  likelihood. 
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Figure  4.  Training  Type  as  a  Function  of  Location  and  Internet  Access.  Dashed  lines  indicate 
reduced  likelihood  of  training  type.  Absent  lines  indicate  zero  likelihood  of  training  type. 
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General  Overview  of  TA  Training  Products.  Several  principles  exist  to  support  the  design 
of  interactive  computer-based  instruction  and  should  be  applied  to  the  TA  authoring  output.  The 
principles  address  interface  characteristics  to  enhance  usability,  utility,  and  attractiveness  and 
instructional  design  features  that  promote  self-directed  learning,  reflective  practice, 
collaboration,  and  retention.  The  key  elements  of  TA  training  products  therefore  should  include 
(a)  advanced  organization  of  instructional  content  (including  learning  objectives,  overview  of  the 
training  product,  etc.);  (b)  a  doctrine-based  tutorial  on  the  basic  training/educational  concepts;  (c) 
expert  perspectives  on  the  learning  topic;  (d)  alternative  approaches  to  executing  the  task  to  be 
learned  (i.e.,  demonstrations  of  the  task  to  be  trained  under  varying  conditions);  (e)  a  summary  of 
the  learning  material  presented  in  the  tutorial,  expert  perspectives,  and  alternative  approaches; 
and  (f)  an  interactive  practical  exercise  (or  set  of  exercises)  for  the  learner  to  conduct  for 
practice,  assessment,  and  feedback  (Hays  et  al.,  2005;  Herrington  &  Oliver,  1995;  Park  & 
Hannafin,  1993). 

Creating  Practical  Exercises.  Based  on  the  Phase  I  findings  described  previously,  the 
focus  of  the  TA  training  authoring  function  should  be  on  the  development  of  scenario-based 
practical  exercises.  Two  major  components  of  practical  exercises  must  be  authored-scenario 
content  and  interactivity. 

Scenario  content  should  be  authored  using  an  interface  in  which  trainers  can  insert  text, 
photos,  video,  and  audio  to  create  a  dynamic  presentation.  Addition  and  modification  of  training 
content  should  be  achieved  via  direct  manipulation  so  that  trainers  may  see  the  immediate  effect 
of  their  actions  on  the  training  product.  This  component  of  the  authoring  capability  would  enable 
trainers  to  decide  the  degree  of  technical  sophistication  they  wish  to  apply  to  their  training 
exercise,  ranging  from  text  only  to  full-scale  multimedia. 

Interactivity  should  be  authored  using  one  of  several  standardized  activity  templates. 
Templates  facilitate  authoring  by  providing  a  means  to  focus  and  constrain  author  actions  in  a 
way  that  produces  useful,  standardized,  and  structured  training.  Templates  also  allow  instructors 
with  little  or  no  programming  skill  to  easily  create  interactive  exercises.  The  following  templates 
were  identified  to  cover  a  wide  range  of  learning  objectives  (LOs): 

•  Overlay  Creation-An  exercise  that  requires  the  trainee  to  drag  and  drop  icons  and/or 
draw  figures  on  specific  locations  on  a  background  graphic  (e.g.,  map,  sketch,  photo, 
etc.).  General  feedback  from  the  trainer  is  provided  and  the  trainee  self-evaluates  the 
correctness  of  his/her  answer. 

•  Test  Question- An  exercise  that  requires  the  trainee  to  answer  one  or  more  multiple- 
choice,  true/false,  or  “select  all  that  apply”  questions.  Student  answers  are  assessed 
automatically  and  explanatory  feedback  written  by  the  trainer  is  displayed. 

•  Self-Evaluation-An  exercise  that  requires  the  trainee  to  create  a  free-text  response  to  an 
open-ended  scenario  provided  by  the  trainer.  General  feedback  from  the  trainer  is 
provided  and  the  trainee  self-evaluates  the  correctness  of  his/her  answer. 

•  Synchronous  Collaboration-An  exercise  that  requires  multiple  trainees  to 
move/add/remove  icons  on  a  shared  whiteboard.  This  template  does  not  support  the 
provision  of  feedback,  but  rather  is  designed  to  provide  a  simple  means  for  conducting 
collaborative  rehearsal. 
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•  Situational  Judgment-An  exercise  that  requires  the  trainee  to  review  a  situation  and  rate 
the  quality  of  a  number  of  ways  of  handling  the  situation.  This  template  is  particularly 
useful  for  exercising  trainees’  ability  to  think  through  “non-tactical”  problems  (e.g., 
dealing  with  uncooperative  civilians)  that  do  not  have  a  single  correct  answer  or  where  no 
appealing  answer  is  available.  Feedback  from  the  TA  involves  showing  a  trainee’s  ratings 
compared  to  those  of  the  trainer  and  other  trainees. 

•  Location  Selection-An  exercise  that  requires  the  trainee  to  point  and  click  to  points  on  a 
2D  graphic  (e.g.,  map,  photo,  sketch,  etc.)  to  demonstrate  knowledge  of  where  things  are 
or  should  be  located.  Trainees  receive  visual  feedback  on  the  points  that  were  not 
correctly  identified,  along  with  the  trainer’s  overall  rationale  for  the  preferred  solution. 

The  TA  must  have  an  architecture  that  links  template  types  to  the  trainer’s  development 
objectives.  The  present  research  indicated  such  an  architecture  must  allow  flexible  entry  into  the 
system  (e.g.,  by  selecting  a  doctrinal  task  or  a  general  competency)  and  must  fit  with  user 
expectations  regarding  the  structure  of  training  exercises  (i.e.,  that  they  are  situated  in  a  specific 
task  context  and  have  explicit  learning  objectives.) 

To  meet  these  requirements,  a  background  task  can  be  used  to  situate  one  or  more 
learning  objectives.  In  this  way,  trainers  can  build  multiple  practical  exercises  using  the  same 
background  materials.  This  format  also  enables  deliberate  practice  of  selected  elements  of  larger 
training  tasks  by  breaking  the  tasks  down  into  their  component  parts.  Recommended  background 
tasks  include: 

•  Conduct  a  route  reconnaissance  (also  tenned  a  route  reconnaissance  patrol); 

•  Establish  an  observation  post; 

•  Reconnoiter  a  built  up  area  (also  known  as  urban  reconnaissance); 

•  Assault  a  building; 

•  Conduct  a  presence  patrol; 

•  Conduct  a  raid  (also  known  as  a  type  of  combat  patrol); 

•  Conduct  a  security  patrol; 

•  Conduct  convoy  escort; 

•  Establish  a  checkpoint  or  traffic  control  point; 

•  Search  a  building; 

•  Secure  a  route  (also  known  as  IED  patrolling); 

•  Cordon  and  knock  or  Cordon  and  search; 

•  Conduct  an  attack; 

•  Conduct  a  defense;  and 

•  Conduct  a  movement  to  contact. 

These  background  tasks  were  selected  based  on  (a)  the  frequency  of  their  occurrence  as 
reported  by  Operation  Iraqi  Freedom  and  Operation  Enduring  Freedom  combat  veterans;  (b) 
their  centrality  in  pre-deployment  training  conducted  at  the  Joint  Readiness  Training  Center;  (c) 
their  representativeness  of  full-spectrum  operations  tasks;  and  (d)  their  presence  in  emerging 
maneuver  doctrine  (e.g.,  DA,  2002b). 
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Recommended  learning  objectives  were  derived  by  reviewing  doctrinal  training  plans 
(e.g.,  DA,  2003)  and  selecting  perfonnance  measures  and/or  supporting  collective  tasks 
associated  with  each  background  task  listed  above.  Performance  measures  or  collective  tasks 
were  not  selected  if  they  were  (a)  difficult  to  address  with  interactive  multimedia  instruction  or 
better  addressed  using  some  other  medium;  or  (b)  not  strongly  associated  with  common  training 
challenges  experienced  at  home  and  in  the  field.  Selected  learning  objectives  are  shown  in 
Appendix  B,  which  presents  a  complete  list  of  each  of  the  background  tasks  and  their  associated 
LOs.  For  trainers  who  conceptualize  their  developmental  goals  in  more  abstract  terms,  LOs  may 
also  be  associated  with  general  competencies  (see  Appendix  C). 

To  summarize,  each  learning  objective  is  associated  with  a  particular  activity  template 
(described  above).  Trainers  may  arrive  at  a  learning  objective  and  associated  activity  template  by 
first  selecting  a  background  task  or  a  general  competency.  As  noted  in  the  next  section,  templates 
associated  with  a  particular  combination  of  background  task  and  learning  objective(s)  should 
feature  generic  content  to  accelerate  the  authoring  process.  Authors  may  also  choose  to  introduce 
a  new  background  task  and/or  learning  objective,  but  generic  content  cannot  be  supplied  for 
novel  task- learning  objective  combination  templates. 

Enhancing  Training  Development.  To  be  useful,  the  TA  must  support  more  rapid 
generation  of  better  training  content.  The  training  creation  process  could  facilitate  the  rapid 
development  of  targeted  training  packages  and  exercises  through  (a)  the  provision  of  generic, 
modifiable  content;  and  (b)  the  ability  to  search  for  up-to-date  training  artifacts  and  infonnation 
on  trends  in  the  operational  environment.  Resources  that  the  TA  can  provide  to  trainers  for 
enhancing  training  development  should  include  - 

•  Reference  materials  (e.g.,  doctrine,  professional  articles,  discussion  posts,  other  archived 
exercises,  existing  computer  based  training  (CBT)  related  to  the  training  topic 

•  A  searchable  database  of  training  artifacts  (including  photos,  maps,  documents, 
illustrations,  video,  etc.) 

•  Generic  content  [e.g.,  training  advance  organizers  with  objectives,  topic  tutorials, 
practical  exercise  descriptions,  etc.,  and  practical  exercise  materials  (e.g.,  scenario 
content,  generic  rules  of  engagement,  signal  operating  instructions,  etc.)] 

Phasell  Development  -  Overview  of  the  Tool 

The  TA  was  designed  to  serve  as  a  training  delivery  tool,  an  authoring  tool,  and  a 
learning  management  system  (LMS).  This  section  begins  with  a  discussion  of  the  scope  of  the 
TA,  and  then  describes  the  various  features  that  allow  it  to  serve  these  three  functions,  and 
concludes  with  a  detailed  description  of  the  TA  search  feature. 

Scope  Definition  and  Tool  Selection 


System  Programming 

Back-End  Database.  The  TA  system  was  designed  with  a  Microsoft  structured  query 
language  (MS  SQL)  database  to  organize  and  store  the  data.  This  platform  was  chosen  because 
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of  its  scalability  and  fast  data  access.  The  TA  MS  SQL  database  was  designed  to  be  as  flexible  as 
possible,  allowing  data  to  be  extracted  for  many  purposes,  even  those  not  currently  programmed. 
Additionally,  new  tables  and  fields  can  be  added  to  the  database  as  needed  to  support  more 
advanced  functionality,  such  as  the  ability  to  create  classes,  assign  learners  to  classes,  and  assign 
training  to  classes. 

Front-End  Interface.  The  front-end  user  interface  was  created  with  Microsoft  ASP.NET 
active  server  pages,  which  can  easily  access  the  MS  SQL  database  and  present  infonnation  in  a 
user-friendly  manner.  Additionally,  most  of  the  complex  programming  is  run  on  the  server, 
rather  than  the  user’s  machine,  leading  to  faster  and  more  consistent  system  performance. 

Multimedia 

The  Phase  II  behavioral  research  identified  a  number  of  types  of  multimedia  (e.g., 
images,  video,  and  audio)  that  would  be  beneficial  to  incorporate  into  the  TA.  However,  due  to  a 
number  of  factors,  the  team  decided  to  focus  primarily  on  images.  The  following  access  and 
usability  factors  played  into  this  decision: 

•  Images  are  the  most  widely  used  and  available  type  of  multimedia; 

•  Most  trainers  have  the  knowledge  and  ability  to  create  images,  while  the  same  is  not  true 

of  video  and  audio;  and 

•  Images  are  much  smaller  in  file  size,  leading  to  faster  downloads  and  training  that  is 

accessible  to  users  with  varying  Internet  connection  speeds. 

Although  it  is  not  currently  possible  to  incorporate  videos  into  training,  videos  were 
included  in  the  search  feature  so  that  they  will  be  available  for  incorporation  when  programming 
is  added  during  future  development  cycles. 

Activity  Templates 

Time  constraints  prevented  the  development  of  all  six  of  the  identified  activity  templates. 
Therefore,  three  templates  -  overlay  creation,  test  questions,  and  self-evaluation  -  were  chosen 
for  development  based  on  their  wide  applicability  (e.g.,  test  questions  and  self-evaluation)  or 
potential  to  add  unique  capabilities  to  the  training  (e.g.,  overlay  creation). 

Overlay  Creation  Tool  Selection.  Selection  of  the  tool  that  would  power  the  overlay 
creation  template  was  an  important  decision.  Many  users  on  Army  networks  do  not  have  the 
ability  to  download  or  install  additional  software.  Therefore,  the  overlay  tool  needed  to  be 
available  from  a  Web  browser  and  require  no  additional  software  installation.  A  number  of  tools 
and  technologies  were  investigated,  but  it  was  ultimately  determined  that  a  Macromedia  Flash- 
based  tool  would  be  the  best  answer,  as  Flash  Player  is  available  through  a  Web  browser  and  is  a 
standard  install  on  all  Army  computers.  Additionally,  Flash  Media  Server,  which  allows 
synchronous  access  to  the  same  data  from  multiple  computers,  was  incorporated  so  that  the  tool 
can  also  be  used  to  support  the  synchronous  collaboration  template  when  it  is  created  in  later 
development  phases. 
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Content  Delivery  Tool  Features 


The  TA  allows  users  to  view  and  complete  training  that  has  been  created  in  the  system. 
Upon  login,  all  users  are  taken  to  the  “My  Training”  page.  This  page  looks  slightly  different  for 
each  user,  but  it  always  contains  a  table  of  training  modules  that  the  user  has  been  assigned  to 
complete.  For  each  module,  the  author  and  date  of  last  access  are  displayed.  A  user  must  click 
the  title  of  the  training  to  be  taken  to  its  table  of  contents.  From  there,  the  user  can  select  a 
section  (e.g.,  introduction,  basic  tutorial,  practice  exercise,  etc.)  and  proceed  through  the  course. 
Each  section  of  training  is  described  below  in  the  discussion  of  TA  authoring  features. 

Authoring  Tool  Features 


Course  Creation 

As  described  above,  the  Phase  II  behavioral  research  identified  fifteen  doctrinal 
background  tasks  and  thirteen  skills  for  which  training  might  commonly  be  developed  by  TA 
users.  These  doctrinal  tasks  and  general  skills  allow  trainers  to  begin  training  development  using 
a  framework  that  is  familiar  to  them.  Trainers  have  two  options  from  which  to  choose  when 
creating  new  training:  to  use  or  modify  existing  training  or  to  create  new  training  from  scratch. 
To  adopt  the  first  approach,  users  can  search  the  TA  to  determine  whether  other  trainers  have 
created  training  on  the  same  task  and  LO.  If  a  user  finds  existing  training  that  is  applicable,  they 
can  create  a  copy  and  use  it  as  the  base  for  his/her  course  (Figure  5). 
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Figure  5.  “Copy  Existing  Training”  Function.  If  trainers  find  existing  training  that  meets  their 
needs,  they  can  copy  that  training  and  use  it  as  the  foundation  for  their  own  course.  In  this 
example,  a  link  to  “Copy  to  My  Training”  is  available  for  each  of  the  existing  courses. 
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When  creating  new  training  from  scratch,  TA  users  are  presented  with  either  doctrinal 
tasks  or  the  general  skills  from  which  they  will  make  a  selection  (Figure  6).  As  described  above, 
the  TA  architecture  links  each  doctrinal  task  and  general  skill  to  multiple  associated  LOs. 
Trainers  can  select  one  or  more  LOs  for  their  training.  Once  the  task  and  LOs  have  been 
selected,  trainers  confirm  their  selections  and  choose  a  title  for  their  training  (Figure  7).  Once 
they  click  the  “Save”  button,  new  training  is  created. 

All  training  created  from  the  pre-defined  tasks  and  LOs  has  a  default  structure  so  that  TA 
users  can  rapidly  create  pedagogically  sound  training  (Figure  8).  This  structure  consists  of  an 
introduction  and  summary  for  the  training  module  and  then  specific  sections  for  each  LO:  “Basic 
Tutorial,”  “What  Right  Looks  Like,”  “Modified  Approaches,”  and  “Practical  Exercise.”  The 
Basic  Tutorial  section  provides  a  general  overview  of  the  LO.  The  “What  Right  Looks  Like” 
section  presents  an  open-ended  scenario  and  demonstrates  a  correct  approach  to  that  scenario 
through  text  and  images.  The  “Modified  Approaches”  section  details  situations  where  Soldiers’ 
approaches  may  change  based  on  changing  environmental  or  situational  factors.  Finally,  the 
“Practical  Exercise”  section  gives  the  learners  a  chance  to  apply  what  they  have  learned  by 
completing  a  scenario-based  exercise.  The  learning  activity  type  represented  in  the  practical 
exercise  is  dependent  on  the  LO.  The  association  between  LO  and  practical  exercise  learning 
activity  was  made  to  maximize  the  similarity  between  learner  interaction  with  the  training 
content  and  the  behavioral  requirements  of  the  operational  setting. 
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Figure  6.  Selecting  a  Task  for  a  New  Training  Module.  When  creating  new  training  from  scratch, 
trainers  can  select  from  doctrinal  tasks  and  general  skills.  In  the  example  presented  here,  a  trainer 
has  selected  the  doctrinal  task  “Reconnoiter  a  Built-Up  Area  or  Urban  Reconnaissance.” 
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Reconnoiter  a  Built-up  Area  or  Urban  Reconnaissance 


a 
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Submit 
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•  Consolidate  and  reorganize 

•  Interact  with  local  nationals 


Save  Training  As  Reconnoiter  a  Built-up  Area  or  Urban  Reconnaissance  T raining 
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Figure  7.  Creating  a  New  Training  Module  with  Three  Learning  Objectives.  The  TA  architecture 
links  each  doctrinal  task  and  general  skill  to  multiple  associated  learning  objectives,  so  that 


trainers  can  select  one  or  more  learning  objectives  for  their  training.  In  this  example,  a  trainer  has 
chosen  to  develop  doctrinal  training  and  has  selected  three  learning  objectives  (i.e.,  Adjust  a  plan 
based  on  new  intelligence;  Consolidate  and  reorganize;  Interact  with  local  nationals).  He  has 
titled  the  training  “Reconnoiter  a  Built-Up  Area  or  Urban  Reconnaissance  Training.” 
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Course  Menu 

Introduction 

□  Adjust  a  plan  based  on  new  intelligence 

Basic  Tutorial 
What  Right  Looks  Like 

Modified  Approaches 

0  Practical  Exercise 
E  Consolidate  and  reorganize 
Basic  Tutorial 
What  Right  Looks  Like 

Modified  Approaches 

E  Practical  Exercise 
B  Interact  with  local  nationals 
Basic  Tutorial 
What  Right  Looks  Like 

Modified  Approaches 

E  Practical  Exercise 
Summary 


Figure  8.  The  Automatically-Generated  Course  Menu  for  a  New  Training  Module.  All  training 
created  from  the  pre-defined  tasks  and  learning  objectives  has  a  default  structure.  This  structure 
consists  of  an  introduction  and  summary  for  the  training  module,  as  well  as  specific  sections 
(i.e.,  “Basic  Tutorial,”  “What  Right  Looks  Like,”  “Modified  Approaches,”  and  “Practical 
Exercise”)  for  each  learning  objective.  In  the  example  shown  here,  the  default  structure  for 
“Reconnoiter  a  Built-Up  Area  or  Urban  Reconnaissance”  and  the  three  selected  learning 
objectives  are  shown. 

Base  Content 

In  addition  to  the  default  course  structure,  which  helps  ensure  that  TA  users  generate 
comprehensive  training,  the  TA  also  generates  default  pages  within  each  section.  These  default 
pages  contain  what  is  called  “base”  or  “generic”  content,  meant  to  be  widely  applicable  to  many 
training  situations.  The  intent  of  the  base  content  is  to  accelerate  the  training  development 
process.  During  the  Phase  II  development,  base  content  was  created  for  two  tasks  (Route 
Security  and  Building  Assault)  and  three  learning  objectives  (Plan  a  Mission  -  Conduct  a  Map 
Reconnaissance/Establish  Control  Measures,  Plan  a  Mission  -  Conduct  an  Attack,  and 
Determine  the  Purpose  and  Effect  of  Indirect  Fires).  However,  any  new  content  added  by  trainers 
and  made  public  is  searchable  and  usable  by  other  trainers.  In  this  way,  the  base  content  will 
become  richer  and  more  robust  as  the  system  is  used. 
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Editing  Static  Pages 

Even  though  the  TA  provides  base  training  content,  the  trainer  has  complete  control  over 
the  content  and  can  edit  or  delete  anything  that  is  not  applicable  to  her  training.  When  viewing 
the  course,  the  trainer  will  see  an  “Edit”  button  in  the  upper  right  comer  of  each  page  (Figure  9). 
Clicking  this  button  will  take  the  trainer  into  edit  mode,  where  they  will  have  the  option  to  edit 
the  text  and  images  appearing  on  the  page. 
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As  the  platoon  leader,  you  must  analyze  the  terrain  for  routes  that  provide  protection  from  direct  and 
indirect  fires  and  from  ground  and  aerial  observation.  The  routes  you  choose  should  facilitate  mission 
accomplishment  within  the  limitations  of  boundaries  and  allow  freedom  of  maneuver 

You  will  receive  the  obstacle  overlay  and  the  situational  template  overlay  from  the  commander  to 
identify  reported  enemy  and  obstacle  locations  You  will  also  receive  the  commander’s  operations 
overlay  to  identify  control  measures  impacting  route  planning  Upon  receipt  of  these  materials,  you 
will  plan  your  route 
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Figure  9.  The  Page  “Edit”  Button.  When  using  the  TA,  trainers  have  complete  control  over 
course  content.  If  they  find  content  that  they  wish  to  edit  or  delete,  they  simply  click  on  the 
“Edit”  button,  which  appears  in  the  upper  right  corner  of  every  page. 

Text  is  edited  through  a  simple  text-editor  that  appears  inline  on  the  page  (Figure  10). 
The  editor  contains  easily  recognized  icons  allowing  the  trainer  to  change  the  font,  size,  color, 
alignment,  and  other  properties  of  the  text.  The  text-editor  also  allows  the  trainer  to  insert  a 
hyperlink  to  another  website  (Figure  1 1)  or  search  for  content  using  the  built-in  search  feature, 
which  is  described  in  a  later  section  of  this  report. 
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Figure  10.  The  Inline  Text-Editor.  Easily  recognized  icons  allow  trainers  to  change  the  font,  size, 
color,  alignment,  and  other  properties  of  the  text. 
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Inserting  a  Hyperlink.  Clicking  on  the  “Link”  button  in  the  text-editor  allows  trainers 
insert  a  hyperlink  to  another  website. 
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When  editing  images,  the  trainer  has  multiple  options.  First,  the  trainer  can  choose  to 
modify  the  current  image  by  changing  its  size  or  adding  a  caption  (Figure  12).  Second,  the 
trainer  can  upload  a  new  image  from  his/her  computer  and  specify  properties  such  as  size, 
caption,  image  credit,  and  whether  the  image  is  searchable  and  usable  by  other  trainers  (Figure 
13).  Finally,  the  trainer  can  use  the  built-in  search  feature  to  search  for  an  image  on  the  Internet 
or  in  other  training  created  using  the  TA. 


Change/Set  Image  Properties 


Size 


Size  image  should  appear  in  course 

Link  to  Full-Size  Image? 

Clicking  on  the  image: 

®  Opens  the  full-size  image  in  a  pop-up  window 
Does  nothing 

Caption 

If  you  would  like  a  caption  to  appear  beneath  your  image,  enter  it  below. 


Save  &  Close  Cancel 

Figure  12.  Setting  Image  Properties  for  an  Existing  Image.  Trainers  can  modify  an  existing 
image  by  changing  its  size  or  adding  a  caption.  In  this  example,  a  trainer  has  set  the  image  size  at 
“Quarter-page  width”  and  has  allowed  for  full-size  images  in  pop-up  windows. 


1}  Quarter-page  width  |^~1; 

Full-page  width 
Half-paqe  width 

Quarter-pa 

Thumbnail 
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Clicking  on  the  image: 

«  Opens  the  full-size  image  in  a  pop-up  window 
Does  nothing 

Caption 

If  you  would  like  a  caption  to  appear  beneath  your  image,  enter  it  below. 


Credit 

If  this  image  does  not  belong  to  you,  please  enter  the  name  of  the  owner. 

Public  Image? 

Would  you  like  to  make  this  image  available  for  use  by  all  Army  Training  Assistant  users  via  the  search  feature? 

®  No 
Yes 

Save  &  Close  Cancel 

Figure  13.  Adding  Image  Properties  to  an  Uploaded  Image.  Trainers  can  upload  images  from 
their  computers  and  specify  properties  such  as  size,  caption,  image  credit,  and  whether  the  image 
is  searchable  and  usable  by  other  trainers. 

Editing  Activity  Pages 

Each  practical  exercise  contains  one  or  more  activity  pages,  or  templates.  The  necessary 
activity  templates  were  determined  through  the  Phase  II  behavioral  research  and  were  designed 
to  reinforce  learning  by  allowing  learners  to  interact  with  the  training  content.  Three  of  the  six 
activity  templates  identified  by  the  research  have  been  developed:  Overlay  Creation,  Test 
Questions,  and  Self  Evaluation.  Each  template  allows  the  trainers  to  ask  questions,  provide 
possible  responses,  and  incorporate  feedback  for  the  learners.  The  difference  between  templates 
is  how  the  learners  interact  with  the  content  to  provide  their  answers. 

Map  Overlay  Creation.  The  map  overlay  creation  template  allows  learners  to  draw  their 
own  map  overlays  based  on  a  scenario  written  by  the  trainer.  The  trainer  provides  a  base  overlay, 
containing  the  map  and  any  other  symbols  that  should  be  present  to  describe  the  situation  (Figure 

14) .  The  trainer  then  provides  an  answer  overlay,  which  depicts  the  trainer’s  “preferred”  version 
of  the  overlay,  along  with  an  explanation  of  why  each  element  was  placed  where  it  was  (Figure 

15) .  When  learners  visit  the  activity  page,  they  see  the  base  overlay  and  a  drawing  interface  that 
allows  them  to  add  their  own  lines,  text,  and  symbols  (Figures  16-19).  Using  the  scenario  as  a 
guide,  learners  create  their  own  map  overlay.  Once  they  are  satisfied  with  their  overlay,  they 
submit  their  answers.  Upon  clicking  submit,  the  trainer’s  answer  overlay  appears,  along  with  an 
explanation  for  the  trainer’s  choices.  The  learners  can  then  compare  their  overlays  with  the 
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trainer’s  overlay.  All  learner  overlays  are  saved  in  the  system  so  that  the  trainer  can  review  them 
at  a  later  time. 


Overlay  Layers  Background  Shapes  Author 


Currently  Editable  Overlay:  I_J Lock  All  Layers  !  Layer  Properties 
Base  Overlay  *  I  Erase  All  Layers  , 

Figure  14.  Inserting  or  Changing  the  Properties  of  the  Map  Background  Image.  In  the  example 
illustrated  here,  a  trainer  has  clicked  on  the  “Map  Properties”  button  at  the  bottom  of  the  page, 
which  will  allow  him  to  insert  an  image,  remove  an  image,  or  adjust  image  scale  and  brightness 


f'JPan  1  -  Zoom  +■  I  Group 

Map  Properties  Refocus  Stage  |  Ungroup 
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Figure  15.  The  Military  Symbol  Editor.  In  this  example,  a  trainer  has  clicked  on  the  “Military 
Symbols”  button  (second  button  down,  from  top  left  of  page),  which  will  allow  him  to  insert  and 
edit  military  symbols.  Trainers  can  also  edit  the  color,  fill,  and  text  associated  with  military 
symbols. 
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Color  Style  Control  Measures 


Overlay  Layers 


Reset  to  Base 


Background 


Shapes  Author 


{7}  /  + 


Group 

Ungroup 


Currently  Editable  Overlay:  MLock  All  Layers  Layer  Properties  I  |  f'JIPan  •  Zoom  + 

Route  *  ’  Erase  All  Layers  |  |  Map  Properties  !  Refocus  Stage 


Figure  16.  The  Control  Measures  Tool.  In  the  example  presented  here,  a  trainer  has  clicked  on 
the  “Control  Measures”  button  (third  button  down,  from  top  left  of  page),  which  will  allow  him 
to  insert  control  measures  such  as  axis  of  advance,  fix,  disrupt,  bridges,  and  bypasses.  Trainers 
can  also  edit  the  color  and  style  of  control  measures. 
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Overlay  Layers  Background  Shapes  Author 


Currently  Editable  Overlay: 

Lock  All  Layers 

J  Layer  Properties 

fJPan  -  Zoom  l  *  I 

Group 

|  Reset  to  Base 

Route 

|  Erase  All  Layers 

|  Map  Properties  Refocus  Stage 

|  Ungroup  | 

Figure  17.  The  Text  Tool.  In  this  example,  a  trainer  has  clicked  on  the  “Text  Tool”  button  (fourth 
button  down,  from  top  left  of  page),  which  will  allow  him  to  insert  and  edit  text.  Trainers  can 
change  the  background  color,  size,  style  and  alignment  of  text  on  their  overlays. 
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Color  Width  Style 


Overlay  Layers 


Currently  Editable  Overlay:  QLock  All  Layers  Layer  Properties 
Route  *  !  Erase  All  Layers 


Background  Shapes  Author 


fJlPan  -  Zoom  +  I  I  Group  |  I  ResettoBase 

Map  Properties  Refocus  Stage  |  Ungroup  | 


Figure  18.  The  Line  Tool.  In  the  example  shown  here,  a  trainer  has  clicked  on  the  “Line  Tool” 
button  (sixth  button  down,  from  top  left  of  page),  which  will  allow  him  to  draw  and  edit  lines. 
Trainers  can  change  the  color,  width,  and  style  of  lines  on  their  overlays. 
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Color  Width  Style 
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Shapes 


Author 


Currently  Editable  Overlay:  nLock  All  Layers  Layer  Properties  I  |  fJlPan  1,  -  Zoom  +  I  Group 
Route  *  Erase  All  Layers  |  !  Map  Properties  Refocus  Stage  |  Ungroup 


Reset  to  Base 


Figure  19.  The  Overlay  Properties  Window.  In  this  example,  a  trainer  has  clicked  on  the  “Layer 
Properties”  button  (bottom  center  of  page),  which  will  allow  him  to  rename  and  re-stack  layers. 


Test  Questions.  The  test  questions  template  allows  the  trainer  to  create  multiple-choice, 
true/false,  or  “select  all  that  apply”  questions  to  test  learners’  knowledge  (Figure  20).  In  addition 
to  indicating  the  correct  answer,  the  trainer  has  the  opportunity  to  provide  custom  feedback 
explaining  to  the  learner  why  her  choice  was  or  was  not  correct.  The  system  evaluates  learners’ 
responses  and  saves  them  so  that  they  can  be  reviewed  by  the  trainer  at  a  later  time  (Figure  21). 
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Figure  20.  Editing  a  Test  Question.  The  test  questions  template  allows  trainers  to  create  multiple 
choice,  true/false,  or  “select  all  that  apply”  questions  to  test  learners’  knowledge.  In  the  example 
presented  here,  a  trainer  has  selected  a  multiple  choice  question  with  four  answers  to  choose 
from.  He  has  also  identified  the  correct  answer,  and  added  feedback  for  correct  and  incorrect 
responses. 
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Figure  21.  Evaluation  of  Learner’s  Response  and  Display  of  Trainer’s  Feedback.  In  addition  to 
indicating  the  correct  answer,  trainers  have  the  opportunity  to  provide  custom  feedback 
explaining  to  learners  why  their  choices  were  or  were  not  correct.  Additionally,  the  system 
evaluates  the  learners’  responses  and  saves  them  so  that  they  can  be  reviewed  by  trainers  at  a 
later  time.  In  this  example,  a  learner  has  selected  the  correct  answer  to  a  multiple  choice 
question,  and  the  feedback  for  that  correct  answer  is  shown. 

Self-Evaluation.  The  self-evaluation  template  is  designed  to  give  learners  the  ability  to 
answer  an  open-ended,  free-response  question  posed  by  the  trainer.  The  system  does  not  evaluate 
the  learners’  answers,  but  instead  the  learners  compare  their  answers  to  the  answers  provided  by 
the  trainer  (Figure  22).  As  with  other  activity  templates,  the  learners’  responses  can  be  saved  for 
later  review  by  the  instructor. 

The  self-evaluation  template  is  designed  to  give  learners  the  ability  to  answer  an  open-ended, 
free-response  question  posed  by  the  trainer.  The  system  does  not  evaluate  the  learners’  answers, 
but  instead  the  learners  compare  their  answers  to  the  answers  provided  by  the  trainer  (Figure  22). 
As  with  other  activity  templates,  the  learners’  responses  can  be  saved  for  later  review  by  the 
instructor. 
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Route  Security:  Coordinate  with  Adjacent  Units 


Plan  a  mission  -  Coordinate  with  adjacent  units  |  Practical  Exercise  |  Self  Evaluation 


Scheme  of  maneuver  was  not  covered  nor  was  there  a  security  plan  indicated 
Iduxing  the  preperatior. .  None  of  the  above  mentioned  are  not  depicted  on  the 
map  with  tne  appropriate  overlays.  Routes  and  routes  of  travel  have  also  been 
onutitted  from  the  plan.  Also,  this  is  a  week  long  mission  which  requires  a 
rest  plan. 


*You  have  already  answered  this  question. 

Feedback 

The  following  list  represents  ways  in  which  the  units  in  the  given  scenario  are  not  coordinated  with  each  other  Compare  your  answer  to  the  list 

•  The  route  security  team  has  established  the  same  communication  frequencies  for  both  combat  patrol  teams  This  will  create  unnecessary  chatter 
and  potentially  confuse  forces  that  may  come  under  attack.  Individual  communication  frequencies  for  each  combat  patrol  team  must  be  established. 

•  The  call  sign  for  combat  patrol  bravo  was  not  established 

•  Fire  support  3  location  was  not  established 

•  The  area  of  coverage  for  fire  support  1  and  2  overlaps  the  designated  patrol  route  for  Combat  Patrol  Alpha  (as  indicated  on  the  corresponding  overlay) 
Coordination  to  establish  a  patrol  route  schedule  for  Combat  Patrol  Alpha  is  of  critical  importance  because  overlapping  fire  support  coverage  may 
result  in  fratricide. 
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Figure  22.  Feedback  Displayed  After  a  Learner  Answers  a  Self-Evaluation  Question.  The  TA’s 
self-evaluation  template  is  designed  to  give  learners  the  ability  to  answer  open-ended,  firee- 
response  questions.  The  system  does  not  evaluate  the  learners’  answers,  but  asks  learners  to 
compare  their  answers  to  those  provided  by  the  trainer.  In  the  example  shown  here,  a  learner  is 
provided  with  the  answers  of  a  trainer  and  asked  to  compare  his  answers  to  the  trainer’s  answers. 


Course-Level  Management 

In  addition  to  supporting  changes  to  content  on  individual  pages,  the  TA  also  provides 
some  course-level  editing  abilities.  First,  trainers  are  able  to  recycle  training  that  is  no  longer 
relevant.  The  training  is  moved  to  the  recycle  bin  (Figure  23),  where  it  can  be  retrieved  for  up  to 
30  days.  At  the  end  of  the  30-day  period,  the  training,  and  any  student  records  associated  with 
the  training,  will  be  deleted.  Additionally,  training  can  be  renamed  at  any  time  by  clicking  the 
“Rename”  button  next  to  the  training  title  on  the  My  Training  page  (Figure  24). 
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Create  New  Training  Recycle  Bin 
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All  courses  in  the  recycle  bin  will  be  permanently  deleted  on  the  delete  date  listed.  To  recover  a  course  from  the  recycle  bin,  click  the  "Recover" 
button. 
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7/06/2008 

|  Recover  ] 
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i  Recover  j 
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TEST  QUESTION  test  training 
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Figure  23.  The  Training  Recycle  Bin.  Trainers  can  recycle  training  that  is  no  longer  relevant. 
When  training  is  moved  to  the  recycle  bin,  it  can  be  retrieved  for  up  to  30  days.  In  this  example, 
training  courses,  their  last  edit  dates,  and  their  delete  dates  are  shown,  along  with  the  “Recover” 
button  for  each  course. 
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Figure  24.  Renaming  a  Training  Module.  From  the  My  Training  page,  training  courses  can  be 
renamed  at  any  time  by  clicking  the  “Rename”  button.  In  the  example  illustrated  here,  a  trainer 
has  clicked  the  “Rename”  button,  and  is  editing  the  name  of  a  course  currently  called  “Conduct 
an  Attack  Training  TEST.” 


Learning  Management  System  (LMS)  Features 


In  addition  to  authoring  features,  the  TA  incorporates  a  number  of  features  usually  found 
in  an  LMS,  such  as  role-based  logins,  training  organization  and  access,  and  the  ability  to  store 
user  data. 


Role-Based  Logins 

Currently,  the  TA  features  three  different  roles:  administrator,  instructor,  and  student. 
The  student  role  is  reserved  for  learners  who  will  be  accessing  content,  but  who  will  not  be 
creating  training.  When  students  access  the  tool,  they  only  see  the  “Training  I’m  Taking”  table 
(Figure  25),  and  do  not  have  any  of  the  training  creation  buttons  along  the  top  of  their  screens. 
Additionally,  when  taking  a  course,  students  do  not  have  the  option  to  edit  the  content  (Figure 
26).  Finally,  when  completing  practice  exercise  activities,  students  are  allowed  only  one  attempt 
and  are  unable  to  answer  any  questions  a  second  time. 
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Figure  25.  Students’  My  Training  Page.  The  student  role  is  reserved  for  learners  who  will  be 
accessing  content,  but  who  will  not  be  creating  training.  In  this  example,  a  student  learner  has 
accessed  the  TA.  Because  he  will  not  be  creating  training,  he  only  sees  the  “Training  I’m 
Taking”  table.  He  does  not  see  any  training  creation  buttons  along  the  top  of  the  screen. 
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Coordinate  With  Adjacent  Units:  Basic  Tutorial  -  Introduction 

The  ultimate  goal  of  adjacent  unit  coordination  is  to  ensure  unity  of  effort  in  the  accomplishment  of 
the  higher  unit's  mission  In  the  context  of  a  route  security  mission,  adjacent  units  may  provide  the 
resources  and  support  necessary  to  reach  the  destination  point  safely  and  efficiently.  Adjacent  unit 
coordination  is  done  face-to-face  when  possible;  however,  the  mission  and  environmental  factors  may 
require  coordination  to  take  place  from  a  remote  location 

Regardless  of  where  coordination  with  adjacent  units  takes  place,  it  is  essential  that  coordination 
address  general  communication  protocols,  unit  positions  routes  of  travel,  location  of  key  points  along 
the  route,  and  fire  support. 


Credit  U.S.  Army  photo  by  Pfc.  Shea  Butler 
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Figure  26.  Students’  Course  Page.  Student  learners  will  not  have  access  to  the  “Edit”  button, 
normally  seen  by  administrators  and  instructors  in  the  upper  right  corner  of  every  page.  In  the 
example  shown  here,  a  student  learner  can  read  a  basic  tutorial  on  route  security,  but  does  not 
have  the  option  to  edit  the  page. 


The  instructor  role  incorporates  all  of  the  features  of  the  student  role  with  the  additional 
ability  to  create,  edit,  rename,  and  delete  content  -  all  of  the  features  discussed  in  the  authoring 
section  above. 


The  administrator  role  incorporates  all  of  the  features  of  the  student  and  instructor  roles, 
but  also  adds  administrative  abilities.  Administrators  have  the  ability  to  approve  new  user 
accounts  (Figure  27).  In  addition  to  approving  user  accounts,  administrators  can  determine  the 
role  into  which  a  new  user  will  be  placed.  The  second  important  ability  that  administrators  have 
is  the  ability  to  assign  training  to  students  (Figure  28).  Once  training  has  been  assigned  to  a 
student,  that  student  will  see  the  training  appear  on  her/his  home  page  when  s/he  logs  in. 
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Figure  27.  Administrators’  Account  Approval  Page.  Administrators  have  the  ability  to  approve 
new  user  accounts  and  determine  the  role  into  which  new  users  are  placed.  In  this  example,  the 
administrator  sees  that  one  person  is  awaiting  approval  for  the  student  role.  By  clicking  on  the 
“Change  Role”  button,  the  administrator  can  also  approve  instructor  and/or  administrator  roles 
for  that  person. 
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Figure  28.  Administrators’  Assign  Training  Page.  Administrators  also  have  the  ability  assign 
training  to  students.  In  the  example  illustrated  here,  the  administrator  views  a  list  of  students  to 
which  he  can  assign  a  course  entitled  “Building  Assault:  Determine  the  Purpose  and  Effect  of 
Indirect  Fires.” 


Search  Feature 

The  TA  offers  trainers  the  ability  to  search  for  content  to  incorporate  into  their  training. 
Access  to  the  search  feature  is  provided  in  both  the  text  and  image  editing  interfaces.  This 
section  discusses  the  search  feature  interface,  as  well  as  the  design  of  the  search  feature.  It 
concludes  by  outlining  the  three  different  ways  in  which  content  is  obtained  when  a  user 
performs  a  search. 

The  Search  Interface 

The  search  interface  was  designed  to  present  search  results  for  multiple  types  of  content 
(text  content,  images,  and  videos)  on  a  single  page,  while  keeping  the  results  organized  by  type 
(see  Figure  29).  This  was  accomplished  through  the  use  of  results  “panels.”  Each  panel  contains 
results  for  one  type  of  content,  and  panels  are  expandable  and  collapsible  to  conserve  screen 
space.  By  modularizing  the  panels,  we  are  able  to  present  an  integrated  view  of  information, 
while  maintaining  a  simple  user  interface.  Within  each  category,  the  TA  search  feature  integrates 
search  results  from  up  to  three  sources  (the  local  TA  database,  the  surface  web,  and  the  deep 
web)  into  a  single  list. 
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Figure  29.  Search  Interface.  The  search  interface  was  designed  to  present  search  results  for 
multiple  types  of  content  on  a  single  page,  while  keeping  the  results  organized  by  type.  Panels 
are  expandable  and  collapsible  to  conserve  screen  space.  In  this  example,  one  can  see  on  the  left 
side  of  the  page  the  “panels”  containing  search  results  for  news,  images,  and  video.  On  the  right 
side  of  the  page,  one  can  see  the  preview  area. 


Each  type  of  content  has  its  own  search  results  presentation.  For  news  and  reference,  the 
title,  date,  and  a  short  snippet  of  text  are  presented,  along  with  any  search  terms  highlighted.  For 
images  and  videos,  thumbnail  images  are  displayed,  along  with  a  title  and  description,  if 
applicable.  At  the  bottom  of  each  panel,  paging  links  are  being  displayed  for  navigational 
purposes.  Additionally,  a  status  indicator  in  each  panel  shows  the  progress  of  the  search  for  that 
panel. 


Entering  Search  Terms.  To  initiate  a  search,  users  can  type  in  their  own  search  terms  or 
use  the  auto-suggestion  feature  to  see  related  keywords  for  the  current  page  (Figure  30).  Once 
the  user  clicks  the  “Search”  button,  the  results  panels  are  populated  with  results.  To  help  increase 
the  relevance  of  a  search,  users  can  choose  to  enable  the  SmartSearch  feature. 
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Figure  30.  Search  Term  Auto-Suggestion.  When  using  the  search  interface,  users  can  type  in 
their  own  search  terms  or  use  the  auto-suggestion  feature  to  see  related  keywords  for  the  current 
page.  In  the  example  presented  here,  a  trainer  views  a  list  of  keywords  generated  by  the  current 
page. 


SmartSearch.  Due  to  the  way  in  which  the  system  is  designed,  users  must  initiate 
searches  from  within  content  pages.  Because  each  page  is  associated  with  a  background  task  and 
a  learning  objective,  the  TA  is  able  to  determine  the  general  type  of  content  for  which  the  user  is 
searching.  As  part  of  the  Phase  II  behavioral  research,  search  keywords  were  identified  and 
linked  to  tasks  and  LOs.  These  default  keywords  can  then  be  used  to  assist  with  a  user-initiated 
search.  “SmartSearch”  may  be  turned  on/off  by  the  user. 

When  SmartSearch  is  enabled,  the  TA  searches  not  only  for  the  user-entered  search 
terms,  but  also  for  the  default  keywords  associated  with  that  page.  In  the  background,  each 
default  keyword  is  sent  to  an  agent  for  searching.  The  user  is  also  able  to  view  the  list  of  default 
keywords  associated  with  the  current  page,  and  can  choose  to  include  or  exclude  each  one  from 
the  SmartSearch  (Figure  3 1).  For  example,  a  user  is  editing  a  page  on  Route  Security  and  enters 
“IED”  as  the  search  term.  The  default  keywords  associated  with  the  route  security  task  include 
“route”  and  “security.”  With  SmartSearch  enabled,  the  TA  will  search  for  results  matching 
“IED,”  “route,”  and  “security,”  giving  those  results  that  match  all  three  a  higher  ranking. 

Without  SmartSearch,  the  TA  will  only  search  for  “IED.”  Therefore,  SmartSearch  has  the  ability 
to  produce  better  ranked  results,  but  the  cost  is  that  the  search  will  take  longer  to  complete  since 
more  terms  are  being  searched. 
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Figure  3 1 .  SmartSearch  Keyword  Selection.  When  SmartSearch  is  enabled,  users  can  view  a  list 
of  default  keywords  associated  with  current  pages,  and  they  can  choose  to  include  or  exclude 
keywords  from  the  SmartSearch.  In  this  example,  a  user  has  chosen  to  include  several  keywords, 
including  “route  clearance,”  “security,”  “route,”  “patrol,”  and  “IED.” 

Search  Result  Rankings  and  Filtering.  Once  results  have  been  obtained  from  each  of  the 
three  sources  (the  local  TA  database,  the  surface  web,  and  the  deep  web),  they  are  integrated  and 
ranked  in  relation  to  one  another.  For  the  news  and  reference  results,  the  ranking  is  based  on  the 
date,  with  more  recent  articles  receiving  a  higher  ranking,  and  also  on  the  degree  to  which  they 
match  the  search  term(s).  For  images  and  videos,  local  TA  results  are  given  the  highest  rankings. 
The  remaining  results  are  ranked  based  on  whether  they  were  acquired  from  a  site  with  a  military 
domain  and  then  based  on  their  original  rankings  on  the  sites  from  which  they  were  obtained. 

The  surface  web  results  are  ranked  according  to  their  term  frequency-inverse  document 
frequency  (TF-IDF)  scores,  as  detailed  in  the  surface  web  search  section  below. 

In  addition  to  ranking  the  search  results,  the  TA  gives  users  the  ability  to  filter  the 
sources  that  are  searched  (Figure  32).  For  example,  a  user  may  wish  to  exclude  local  TA  content 
from  the  search.  Un-checking  the  “TA  Database”  box  in  the  filter  window  will  remove  all  local 
results  from  the  results  panels. 
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Figure  32.  Search  Source  Filtering.  The  TA  gives  users  the  ability  to  filter  the  sources  that  are 
searched.  In  this  example,  a  user  has  chosen  to  exclude  local  TA  content  from  a  search  by  un¬ 
checking  the  Army  Training  Assistant  box  in  the  filter  window. 


The  Preview  Panel.  The  preview  panel  allows  users  to  preview  a  search  result  before 
deciding  whether  to  include  it  in  their  training.  For  news  and  reference  results,  the  preview  panel 
will  allow  the  user  to  open  the  web  page  in  a  new  window.  Within  that  window,  users  can  copy 
any  text  they  wish  to  use  into  a  box  at  the  top  of  that  page.  With  the  click  of  a  button,  that  text 
will  be  copied  to  the  preview  panel  for  insertion  into  the  training.  For  image  results,  the  preview 
panel  will  show  a  small  version  of  the  image  and  will  allow  users  to  specify  settings  for  the 
image  before  inserting  it  into  their  training  (Figure  33). 
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Figure  33.  Search  Image  Preview.  When  conducting  image  searches,  the  preview  panel  will 
show  a  small  version  of  the  image  and  will  allow  users  to  specify  settings  for  the  image  before 
inserting  it  into  their  training.  In  the  example  shown  here,  a  user  has  chosen  for  a  photo  to  be 
displayed  at  quarter-page  width  and  to  appear  in  a  pop-up  window  when  clicked. 


Search  Feature  Design 


The  Phase  I  analysis  of  the  Army  training  process  combined  with  the  Phase  II  behavioral 
research  identified  a  need  for  the  ability  to  search  for  up-to-date  training  artifacts  and 
information  on  trends  in  the  operational  enviromnent.  This  search  capability  would  provide  the 
end  user  the  ability  to: 


•  Search  the  TA  to  retrieve  previously  created  training  content; 

•  Search  an  internal  TA  database  that  maintained  a  collection  of  images,  text,  and  other 
media  assets  that  were  explicitly  placed  in  the  TA  database  as  “relevant”  materials  during 
the  tool’s  development;  and 

•  Search  a  database  that  was  automatically  created  by  an  advanced  web  crawler  that 
collects  pages  on  the  surface  web  meeting  criteria  specified  by  the  needs  analysis  carried 
out  in  the  Phase  I/II  behavioral  research.  The  surface  web  is  primarily  the  set  of  web 
pages  that  exist  on  the  World  Wide  Web  that  can  be  retrieved  without  a  password,  and 
without  typing  a  query  into  a  form  on  a  page.  Surface  web  pages  are  written  in  Hypertext 
Markup  Language  (HTML)  and  are  known  as  HTML  pages.  Google™  and  other  typical 
search  engines  index  the  content  of  static  HTML  pages  and,  when  queried,  provide  a  list 
of  pages  that  meet  the  search  criteria. 
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Specifically,  the  advanced  web  crawler  would  begin  with  a  list  of  “seed”  URLs  provided 
by  the  Phase  II  behavioral  research.  The  crawling  system  would  examine  each  of  those  URLs  to 
determine  whether  the  current  page  should  be  included  in  the  crawler’s  database.  The  system 
would  then  determine  whether  to  include  the  page  or  not  based  on  a  set  of  search  tenns  that  were 
provided  by  the  Phase  II  behavioral  research.  If  the  page  under  consideration  met  the  criteria  for 
inclusion  in  the  database,  it  would  be  stored  and  indexed  with  the  other  included  documents.  The 
crawler  would  then  follow  the  same  procedure  on  each  of  the  links  on  that  document.  If  a  page 
does  not  meet  the  criteria  for  inclusion,  then  it  would  not  be  saved  and  indexed.  However,  the 
crawler  would  examine  any  links  on  that  page,  up  to  four  levels  deep.  That  is,  from  any  given 
page,  the  crawler  would  crawl  each  link  on  that  page,  then  the  links  on  each  of  those  pages,  and 
so  on,  to  a  level  of  four  layers  deep.  If  a  page  was  detennined  to  merit  inclusion  in  the  surface 
web  database,  the  depth  counter  was  reset  at  that  point,  thus  enabling  the  crawler  to  continue 
crawling  down  that  path  until  it  no  longer  found  relevant  pages. 

The  searchable  database  of  documents  can  be  queried  by  an  end  user  via  the  search 
interface  in  the  TA.  Communication  between  the  TA  and  the  surface  web  crawler  is  handled  via 
Extensible  Markup  Language  (XML)  files.  The  XML  allows  for  the  efficient  transfer  of  data  in  a 
format  that  can  be  read  by  many  different  applications,  such  as  the  TA.  The  search  results  are 
translated  from  XML  to  an  easily  readable  format  and  are  displayed  in  the  TA  user  interface. 

Meetings  with  project  sponsors  highlighted  a  need  to  be  able  to  query  “deep  web”  sites, 
as  well  as  monitor  news  feeds  and  other  “up-to-the-minute”  sites  in  order  to  include  those 
resources  in  addition  to,  and  integrated  with,  the  search  results  from  the  above  mentioned 
sources.  The  deep  web  consists  of  those  data  sources  that,  while  available  on  the  World  Wide 
Web,  are  not  searched  by  typical  search  engines  because  the  content  of  the  deep  web  exists  in 
databases,  rather  than  as  static  HTML  pages.  When  a  user  queries  a  deep  web  site,  the  system 
creates  an  HTML  (or  other  document  type)  on  the  fly  that  the  user  can  then  view.  An  example  of 
a  deep  web  site  is  an  airline  reservation  site.  The  airline  reservation  site  has  a  database  of  flight 
times,  prices,  and  other  information.  When  users  type  their  queries  into  the  search  box  on  the 
site,  a  database  query  is  executed,  and  a  customized  page  is  created  that  contains  the  information 
the  customer  requested. 

The  Local  TA  Database  Search 

One  of  the  most  important  capabilities  of  the  TA  is  the  ability  to  allow  trainers  to  access 
and  reuse  existing  training  content  created  by  other  trainers.  Therefore,  when  a  trainer  searches 
for  content,  the  TA  searches  the  local  TA  database  in  addition  to  the  Web.  There  are  two  types  of 
content  for  which  the  local  database  searches:  text  and  images.  Currently,  there  are  not  any 
videos  stored  in  the  local  TA  database. 

Text.  The  local  TA  search  returns  complete  pages  of  content.  To  rank  the  pages  in  order 
of  potential  relevance,  the  system  first  looks  for  the  search  term(s)  in  the  text  of  all  pages  in  the 
system.  The  resulting  pages  are  then  evaluated  to  detennine  whether  they  share  the  same 
background  task  and  learning  objective  as  the  page  from  which  the  search  was  initiated.  Those 
pages  containing  the  search  tenn(s)  and  the  same  task  and  LO  are  given  the  highest  ranking, 
while  those  containing  only  the  search  term(s),  but  not  the  same  task  or  LO  are  given  the  lowest 
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ranking.  Pages  containing  the  search  term(s)  and  either  the  same  task  or  LO  are  given  an 
intermediate  ranking. 

Images.  The  local  TA  search  also  finds  images  stored  within  the  TA  database.  Images  in 
the  TA  image  library,  as  well  as  images  that  have  been  uploaded  by  users  and  made  searchable, 
possess  “tags”  that  describe  the  images.  Therefore,  the  local  TA  search  first  compares  the  search 
tenn(s)  to  the  image  tags.  Images  with  tags  that  match  the  search  tenn(s)  are  given  the  highest 
ranking.  Next,  images  are  pulled  from  pages  that  meet  the  text  search  criteria  described  above 
and  ranked  accordingly  (i.e.,  images  from  pages  with  the  highest  text  ranking  are  given  the 
highest  image  ranking). 

The  Surface  Web  Search 

The  function  of  the  surface  web  search  is  to  create  a  corpus  of  documents  that  are  related 
to  a  wide  array  of  military  training  tasks.  The  primary  goal  of  this  portion  of  the  search  capability 
is  to  have  a  large  resource  of  relevant  documents  on  hand  from  which  instructors  can  draw. 
Results  from  the  surface  web  search  only  fall  into  the  news  and  reference  category. 

In  order  to  create  the  surface  web  crawler,  the  team  chose  between  four  basic  sources  of 
technology: 

1 .  A  commercial  application, 

2.  Open  source  software, 

3.  Extant  software  from  the  National  Center  for  Supercomputing  Applications  (NCSA)  that 

was  built  to  provide  this  capability  for  other  projects,  and 

4.  A  tool  developed  specifically  for  this  project. 

After  evaluating  several  commercial  applications  and  assessing  the  requirements  for  this 
project  against  the  capabilities  of  the  extant  NCSA  software,  commercial  applications,  and  open 
source  applications,  we  chose  to  utilize  open  source  software  for  this  portion  of  the  search 
capability.  Specifically,  we  chose  to  utilize  Nutch  (http://lucene.apache.org/nutch/).  Nutch 
provides  the  capabilities  that  are  required  for  the  project,  is  available  for  both  Windows  and 
Linux  platforms,  and  enabled  us  to  implement  the  surface  web  search  quickly.  The  Nutch 
software  allowed  us  to  implement  our  own  ranking  scheme  for  results  from  the  surface  web 
crawl.  The  algorithm  we  employed  was  term  frequency-inverse  document  frequency  (TF-IDF). 

The  TF-IDF  is  a  weight  that  is  important  in  document  ranking  schemes  in  the  areas  of 
Information  Retrieval  and  Text  Mining.  Tenn-frequency  (TF)  refers  to  the  number  of  times  one 
tenn  appears  in  one  document.  Inverse  document  frequency  (IDF)  is  a  measure  of  the  importance 
of  one  tenn  within  the  corpus  of  documents.  The  TF-IDF  weight  is  determined  by  comparing  the 
number  of  times  a  word  appears  in  a  specific  document  to  the  number  of  times  the  word  appears 
in  the  entire  collection  of  documents.  It  is  often  desirable  to  use  nonnalization  techniques  with 
the  weights.  Common  sense  tells  us  that  if  a  tenn  appears  in  every  document  of  a  corpus,  that 
tenn’s  importance  would  be  less  (e.g.,  “is,”  “to,”  “that,”  etc.)  than  tenns  that  occur  in  only  a  few 
documents.  Hence,  the  TF-IDF  weight  gives  more  importance  to  terms  that  appear  in  only  a  few 
documents.  If  a  term  appears  multiple  times  in  a  few  documents,  those  documents  are  given 
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higher  rankings  than  those  with  less  occurrences  of  that  term.  TF-IDF  has  been  widely  applied  in 
search  engine  scoring  and  ranking. 


The  TA  surface  web  crawling  system  ranks  relevant  documents  for  each  search  query 
using  a  scalar  score  (Salton  &  Buckley,  1988)  computed  via 

score(q, d)  =  coord ( q , d)  ■  qNorm  (q) •  ^  ^  \tf(t,f,d)- idf  (t) 2  ■  boost  f  ■  /Norm  (/, d )]  (E.  1) 

tGq  f  sd 


where  the  query,  q,  is  a  set  of  terms,  /;  the  document,  d,  is  a  set  of  fields,/;  coord(q,d)  is 
representative  of  the  number  of  query  terms  appearing  in  the  document;  qNorm(q)  is  a 
nonnalization  factor  that  ensures  that  the  score  is  independent  of  query  length;  tf(t,fd)  is  the 
frequency  of  term  t  in  field /;  idf(t)  is  the  inverse  frequency  of  tenn  t  in  the  corpus  of  documents; 
boostfis  a  factor  representing  the  relative  importance  of  field / (e.g.,  boostijRL  =  4  and  boostanchor 
=  2);  and  fNorm(f,d)  is  a  normalization  factor  that  ensures  the  score  is  independent  of  document 
length. 


Additionally,  those  results  that  come  from  a  .mil  domain  are  flagged  in  order  to  maintain 
their  place  in  the  search  result  ordering  according  to  relevancy,  but  also  to  allow  them  to  be 
easily  identified  in  the  event  the  searcher  prefers  to  filter  results  to  display  only  those  results 
from  a  military  domain. 

Once  this  surface  web  search  capability  was  in  place,  resources  were  diverted  to  the  deep 
web  capabilities  that  were  not  in  the  original  project  specification,  but  were  identified  as 
important  capabilities  during  subsequent  meetings  with  project  stakeholders. 

The  Deep  Web  Search 

A  deep  web  crawler  can  be  used  to  extract  content  from  the  web  that  is  hidden  behind 
database  forms  where  traditional  web  crawlers  cannot  reach  without  enumerating  all  possible 
inputs.  Much  of  the  media  content,  specifically  images,  we  wanted  to  include  in  the  TA  search  is 
found  in  the  deep  web.  Traditional  web  crawling  for  media  is  not  always  fruitful,  as  some  videos 
and  images  do  not  have  descriptions.  Even  if  descriptions  are  provided,  it  is  difficult  to  associate 
the  description  with  a  particular  image  if  there  are  multiple  images  on  a  page.  However,  many 
images  are  housed  on  web  sites  that  have  their  own  custom  search  features.  On  these  sites,  the 
image  data  is  often  stored  in  a  database,  along  with  image  tags  that  allow  the  site’s  search  feature 
to  accurately  find  an  image. 

To  leverage  each  web  site’s  custom,  built-in  search  feature,  we  used  a  data  extraction 
tool  called  AgentIDE.  AgentIDE  is  a  browser-based  development  environment  that  can  create  a 
wrapper  around  a  web  site,  called  an  agent,  which  can  be  programmed  to  fill  in  web  forms  and 
simulate  mouse  clicks  to  navigate  around  the  site.  AgentIDE  was  developed  by  Cazoodle,  Inc.,  a 
company  which  focuses  on  data  extraction  and  integration  to  create  interesting  vertical 
applications  (e.g.,  an  apartment  search)  by  compiling  data  from  various  sources.  An  individual 
with  average  computer  knowledge  can  create  agents  within  a  short  time  frame,  eliminating  the 
need  for  hiring  skilled  programmers  or  writing  custom  parsers. 
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The  technique  used  for  extraction  is  intuitive  to  people  familiar  with  web  browsers.  A 
user  can  use  a  mouse  to  point  and  click  on  the  visual  elements  on  the  web  page  and  provide  a 
meaningful  example  for  AgentIDE  to  infer  other  records.  For  example  a  user  selects  a  title, 
description,  and  date  for  one  record  and  the  system  infers  other  related  records,  marked  in  green 
and  red  for  distinction  (Figure  34).  AgentIDE  understands  the  visual  relationships  between 
entities  on  the  page  (e.g.,  date,  title,  and  snippets  of  news)  and  can  accurately  associate  them 
together.  If  the  auto-generated  visual  relationships  contain  errors,  a  manual  correction  can  be 
specified  by  the  user. 


Figure  34.  AgentIDE  Visual  Data  Extraction.  AgentIDE  understands  the  visual  relationships 
between  entities  on  the  page  (e.g.,  date,  title,  and  snippets  of  news)  and  can  accurately  associate 
them  together.  In  the  example  shown  here,  a  user  selects  a  title,  description,  and  date  for  one 
record,  and  the  system  infers  other  related  records,  marked  in  green  and  red  for  distinction. 

In  addition  to  extracting  information  from  a  web  page,  an  agent  can  navigate  between 
pages.  This  navigation  is  visually  represented  by  a  state  diagram,  shown  in  the  upper  section  of 
Figure  35.  In  this  example,  the  state  diagram  points  from  an  archived  page  to  a  listing  page  and 
shows  that  the  agent  will  navigate  from  one  to  the  other  by  clicking  a  link.  The  link  is  a  user- 
defined  element  from  the  web  page  (e.g.,  a  next  button)  that  will  facilitate  the  state  transition. 
State  transition  can  mean  a  transition  from  page  to  page,  or  a  transition  between  different 
sections  within  a  page.  With  both  a  navigational  component  and  an  extraction  component, 
AgentIDE  provides  the  flexibility  to  extract  any  type  of  data  on  the  Web. 
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There  are  two  types  of  agents  used  for  crawling.  The  first  type  of  agent  is  a  periodic 
crawler  that  extracts  archived  content  from  news,  image,  and  video  sites.  Depending  on  the  type 
of  content  being  crawled,  different  pieces  of  infonnation  are  extracted.  For  news  and  references, 
the  title,  description,  date,  and  full  content  of  the  article  are  extracted  and  saved.  For  images,  the 
image  unifonn  resource  locator  (URL),  the  image  thumbnail  URL,  the  title,  description,  and  date 
are  extracted.  For  videos,  the  title,  URL,  and  video  time  are  extracted.  All  extracted  information 
is  stored  in  a  separate  table  in  the  MySQL  database  with  a  predefined  schema.  The  agents  are 
built  with  this  schema  in  mind  so  they  can  be  seamlessly  integrated  into  the  system. 

The  second  type  of  agent  is  an  “on-the-fly”  agent  that  helps  the  user  submit  a  query  to  a 
site  and  extract  its  content.  This  extraction  happens  in  real-time  and  can  give  users  a  quick 
response.  The  on-the-fly  agents  use  the  same  data  schema  as  the  periodic  crawler  and  there  is 
virtually  no  difference  between  the  two,  besides  the  amount  of  content  extracted  and  how  the 
execution  is  triggered.  When  a  user  submits  a  search,  the  TA  passes  the  user-entered  search 
tenn(s)  to  the  on-the-fly  agents.  Each  agent  is  tailored  to  a  specific  web  site  and  uses  the  search 
tenn(s)  to  dynamically  query  the  site  using  the  site’s  built-in  search  engine.  The  agents  then 
extract  content  from  the  search  results  page  generated  on  each  site  and  pass  that  information 
back  to  the  TA.  In  Figure  35,  the  agent  is  built  for  the  Wikipedia  web  site  to  extract  content  by 
issuing  a  search  query.  The  agent  has  issued  the  query,  “Iraq  IED,”  and  has  returned  a  set  of 
results  from  which  the  desired  pieces  of  data  will  be  extracted. 

Execution  Engine.  To  support  multiple  agents  in  parallel,  we  developed  a  distributed  job 
dispatching  system  that  can  scale  to  hundreds  of  machines.  The  user  queries  will  trigger  the  job 
dispatching  system  to  distribute  the  jobs  and  gather  results.  The  extracted  content  is  inserted  into 
a  MySQL  database  so  that  it  can  be  indexed  and  immediately  available  for  search  queries.  In 
addition  to  running  agents,  the  execution  engine  supports  data  retrieval  from  web  services.  For 
example,  results  from  the  TA  database  and  surface  web  crawler  are  delivered  to  the  execution 
engine  via  web  services.  The  results  are  then  integrated  with  the  results  from  the  agents. 

In  summary,  the  search  feature  facilitates  the  retrieval  of  related  training  content  and 
resources,  allowing  the  user  to  focus  on  creating  the  training.  The  next  section  presents  the 
method  by  which  testing  and  evaluation  of  the  Army  TA  alpha  release  was  conducted.  The 
results  of  this  evaluation  are  summarized  along  with  their  implications  for  future  research  and 
development  objectives. 
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Figure  35.  Search  Agent  for  Wikipedia.  When  a  user  submits  a  search,  the  TA  passes  the  user- 
entered  search  terms  to  on-the-fly  agents.  Each  agent  is  tailored  to  a  specific  web  site  and  uses 
the  search  terms  to  dynamically  query  the  site  using  the  site’s  built-in  search  engine.  The  agents 
then  extract  content  from  the  search  results  page  generated  on  each  site  and  pass  that  information 
back  to  the  TA.  In  this  example,  the  agent  was  built  for  the  Wikipedia  web  site  to  extract  content 
by  issuing  a  search  query.  The  agent  has  issued  the  query,  “Iraq  IED,”  and  has  returned  a  set  of 
results  from  which  the  desired  pieces  of  data  will  be  extracted. 

Phase  II  Findings 
Method 


Participants 

Thirty-nine  Soldiers  participated  in  alpha  testing:  1 1  non-commissioned  officers  (10 
sergeants  and  1  master  sergeant)  and  28  commissioned  officers  (7  lieutenants,  15  captains,  5 
majors,  and  1  lieutenant  colonel).  All  but  three  of  the  participants  had  deployment  experience, 
with  79%  (N=  31)  having  had  multiple  deployments.  Sixteen  (41%)  participants  held  current 
duty  positions  in  branches  representative  of  the  TA  target  audience  (i.e.,  infantry,  armor, 
cavalry).  Sixteen  participants  tested  the  Map  Overlay  template,  eleven  participants  tested  the 
Test  Question  template,  and  ten  tested  the  Self-Evaluation  template.  Two  participants  partially 
tested  the  Map  Overlay  template. 
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Procedure 


TA  alpha  testing  took  place  during  two  troop  support  (“umbrella”)  weeks,  one  held  at 
Fort  Hood,  TX  (N  =  18)  and  one  at  Fort  Carson,  CO  (N  =  21).  In  both  sessions,  the  templates 
tested  were  Test  Question,  Self-Evaluation,  and  Map  Overlay.  The  protocols  for  the  two  test 
sessions  differed  slightly  due  to  the  scheduled  availability  of  test  participants. 

Sessions  at  Fort  Hood  were  approximately  2  hours  long  and  consisted  of  a  demographic 
survey  (Appendix  D),  followed  by  a  short  demonstration  of  the  TA  capabilities,  and  then  limited, 
guided  user  testing  of  the  TA  authoring  functions  (the  training  generation  testing  protocol  is 
shown  in  Appendix  E).  After  completing  user  testing,  participants  filled  out  a  user  impressions 
survey  (Appendix  F)  and  participated  in  a  focus  group  (protocol  shown  in  Appendix  G). 

Sessions  at  Fort  Carson  lasted  approximately  4  hours  and  also  included  a  demographic 
survey,  user  testing  of  the  TA,  a  follow-up  impressions  survey,  and  a  focus  group.  At  Fort 
Carson,  however,  test  participants  also  completed  a  training  module  as  a  trainee  (usability  testing 
protocol  is  shown  in  Appendix  H).  Additionally,  participants  conducted  more  in-depth  testing  of 
the  authoring  functions  (Appendix  I). 


Results 


General  User  Impressions 

Detailed  User  Impressions  Survey  results  are  shown  in  Appendix  J.  Those  results 
indicated  that  test  participants  generally  found  the  Army  TA  to  be  easy  to  use  (46-86%, 
depending  on  the  feature  assessed)  with  the  majority  reporting  that  they  could  and  would  use  it 
(75%  and  66%  respectively).  Concerns  about  time  constraints  and  computer  access,  particularly 
at  the  small  unit  level,  led  testers  to  recommend  schoolhouse  applications  for  the  tool.  Other 
barriers  to  adoption  that  testers  listed  included  limited  access  to  classified  content  and  the  quality 
of  generic  base  content.  Some  test  participants  suggested  that  the  tool  be  used  to  rapidly  generate 
training  that  does  not  change  frequently  because  taking  the  time  even  to  modify  existing  training 
would  not  be  feasible.  Another  suggestion  was  that  the  tool  would  be  helpful  for  providing 
training  to  people  who  have  changed  branch  and  need  to  get  up  to  speed  on  new  tactics, 
techniques,  and  procedures  (TTP).  Test  participants  seemed  to  agree  that  the  tool  would  be  best 
for  lower-level  leader  training. 

Technical  Issues  Identified 

Consistent  with  the  purpose  of  alpha  testing,  users  also  identified  a  number  of  technical 
issues  with  the  TA  that  represented  the  normal  glitches  in  the  development  process  of  complex 
technologies.  Issues  ranged  from  features  not  displaying  properly  on  the  page,  to  images  not 
saving  to  the  database,  to  code  that  needed  to  be  optimized  for  multiple  simultaneous  users.  With 
such  a  complex  system,  and  so  many  possible  ways  to  accomplish  the  same  task,  the  alpha 
testing  played  a  crucial  role  in  identifying  these  issues. 


54 


Training  Content  Issues  Identified 

Because  the  Fort  Hood  sample  had  a  relatively  low  proportion  of  test  participants  from 
the  intended  target  audience  [i.e.,  1 1%  (2/18)  in  the  combat  arms],  content  quality  was  explicitly 
analyzed  only  by  the  Fort  Carson  sample.  Some  knowledgeable  testers  at  Fort  Hood  made 
comments  about  content,  however.  Probably  the  most  important  observation  was  that  if  the 
content  is  not  considered  effective,  the  tool  will  not  be  perceived  as  useful,  no  matter  how  easy  it 
is  to  use.  Users  would  have  to  change  too  much  in  order  to  create  training,  which  would  lead 
them  to  fall  back  on  current  alternatives. 

This  observation  appeared  to  be  born  out  by  the  Fort  Carson  visit,  which  had  a  larger 
number  of  target  audience  members  and  involved  the  construction  of  actual  courses.  In  general, 
Fort  Carson  testers  indicated  that  the  generic  content  as  currently  developed  was  inadequate. 
They  perceived  it  to  be  too  basic,  too  “civilian,”  too  rigid  (rote  memorization,  text-based  instead 
of  scenario-based  and  interactive),  and  not  relevant  to  or  best  suited  for  emerging  training 
requirements.  Participants  indicated  some  skepticism  that  the  tool  would  be  able  to  do  better 
providing  up-to-date  content  than  other  Army  resources  like  the  Center  for  Army  Lessons 
Learned.  The  matter  of  producing  classified  versus  unclassified  training  came  up  as  an  issue  for 
Fort  Carson  test  participants,  who  generally  reported  that  unclassified  content  was  too  general  to 
be  useful  for  unit  training. 

Usability  Issues  Identified 

My  Training  Page  and  Common  Training  Pages.  Test  participants  generally  found  the 
My  Training  page  very  simple  to  use.  The  only  common  issue  observed  was  that  participants  did 
not  seem  to  grasp  the  distinction  between  “Training  I’m  Taking”  and  “Training  I’ve  Authored.” 
This  issue  may  have  arisen  because  test  participants  relied  on  experimenters  more  so  than 
exploration  to  figure  out  the  interface.  Among  the  common  training  pages,  some  users  confused 
the  Introduction  page  with  other  components  of  the  training  (e.g.,  Introduction,  Basic  Tutorial, 
etc.).  One  user  recommended  using  a  word  other  than  “Introduction”  once  users  are  no  longer  on 
the  introduction  page. 

Basic  Page  (Editing  Text).  Once  reaching  the  text  editor,  testers  (80%)  found  it  simple  to 
modify  generic  content  text.  Difficulties  arose  getting  the  page  into  Edit  mode  and  figuring  out 
how  to  bring  up  the  text  editor  once  in  Edit  mode.  Test  participants  had  to  look  around  for  some 
time  before  finding  the  Edit  button  in  the  upper  right  toolbar.  They  did  not  seem  to  fully 
understand  the  difference  between  Preview  Mode  and  Edit  Mode.  Once  in  Edit  mode,  most  users 
had  trouble  figuring  out  how  to  bring  up  the  text  editor.  When  they  saw  the  gray  box,  they  did 
not  know  that  they  had  to  click  on  it  or  the  edit  icon  in  order  to  start  making  changes.  The 
explanatory  “tool-tip”  only  came  up  after  the  mouse  had  been  still  for  a  relatively  long  period  of 
time. 


Some  users  had  difficulty  figuring  out  how  to  add  a  hotlink  to  the  text.  It  may  be  the  case 
that  they  found  it  difficult  to  imagine  how  a  link  would  be  used  in  an  actual  training  product,  and 
it  was  not  obvious  that  text  had  to  be  highlighted  in  order  to  make  it  a  link.  Once  this  was 
explained,  users  typed  in  text  and  generally  figured  out  how  to  activate  it  as  a  link.  There  was 
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occasional  confusion  between  adding  hyperlinked  text  and  adding  a  URL  to  underlie  the  text. 
Users  had  trouble  with  closing  the  add  link  box  because  they  were  unsure  whether  their  entries 
would  be  saved.  This  confusion  arose  because  the  method  for  closing  the  add  link  box  (an  “x”  in 
the  upper  right  comer)  is  the  typical  convention  for  closing  an  application.  Closing  applications 
does  not  automatically  save  changes,  but  closing  the  add  link  box  does. 

Basic  Page  (Editing  Images) .  Test  participants  (86%)  found  editing  image  properties 
quite  simple,  once  they  got  into  the  image  edit  mode.  Users  had  difficulty,  however,  figuring  out 
how  to  get  into  image  edit  mode.  Under  uncertain  conditions,  users  tried  to  edit  the  image  by  (1) 
clicking  on  the  image;  (2)  right-clicking  on  the  image;  or  (3)  clicking  on  the  Internet  Explorer 
(IE)  icons  that  appear  when  an  image  is  moused  over  (e.g.,  save,  open  My  Images,  etc.). 

Search  Capability.  Test  participants  (65%)  generally  found  the  search  capability  easy  to 
use.  The  search  capability  was  slow,  however,  and  many  of  the  results  were  not  relevant  to  the 
training  topic  at  hand  or  quite  some  time  out  of  date.  When  in  the  search  mode,  users  found  it 
somewhat  unintuitive  to  open  and  close  panels  (and  determine  whether  a  panel  was  open  or 
closed).  Users  did  not  find  retrieving  news  articles  to  be  very  helpful  for  creating  training. 

Instead  they  wanted  restricted  or  classified  content,  such  as  TTP. 

Editing  the  Self-Evaluation  and  Test  Question  Templates.  Test  participants  found  both  the 
Self-Evaluation  template  and  the  Test  Question  template  easy  to  use.  One  common  problem 
encountered  was  that  users  expected  to  be  able  to  save  each  component  of  the  question 
individually  (rather  than  having  to  scroll  to  the  bottom  to  save  the  entire  question).  Users  of  the 
test  question  template  did  have  some  difficulty  understanding  the  different  feedback  types. 
Testers  who  talked  about  feedback  type  indicated  that  they  would  not  give  feedback 
independently  of  student  answers. 

Editing  Map  Overlay  Template.  Editing  the  map  overlay  template  was  a  little  more 
difficult  (46%  rated  it  easy  to  use).  When  editing  the  overlay  template,  testers  had  the  same 
issues  as  testers  of  the  other  templates  with  regard  to  saving  individual  components.  They 
wanted  to  be  able  to  do  this  instead  of  having  to  scroll  down  to  the  bottom  of  the  page  and  save 
the  whole  question.  In  general,  test  participants  seemed  a  little  confused  about  base  overlays 
versus  the  answer  overlay.  They  did  not  seem  to  understand  what  they  were  working  on  - 
question  content  or  answer  key.  There  were  also  a  number  of  usability  issues  identified  with  the 
map  drawing  tool  itself,  as  testers  found  that  some  features  did  not  behave  in  the  way  that  they 
expected.  However,  once  these  features  were  explained,  most  testers  had  no  difficulty  using  the 
tool. 


Discussion 

Test  participants  were  supportive  of  the  TA  concept  and  provided  numerous  constructive 
comments  on  how  to  enhance  the  usability  and  applicability  of  the  tool.  The  key  implications  of 
their  input  for  future  development  can  be  summarized  as  follows: 

1 .  The  TA  generally  is  easy  to  use,  but  it  represents  something  new  to  leam  so  the  degree  to 
which  the  tool  advances  the  training  development  process  will  be  key  to  adoption. 


56 


2.  Training  content  generated  by  the  TA  will  be  most  relevant  to  the  operational 
environment  if  it  is  classified.  This  information  is  the  most  up  to  date  and  the  most 
important  to  deployed  and  pre-deployed  trainees. 

3.  Limited  access  to  classified  computers  makes  it  infeasible  to  deliver  large-scale  classified 
training  with  the  TA. 

4.  The  best  application  environment  for  the  TA  if  classified  content  cannot  be  used  is  the 
schoolhouse,  particularly  for  cadets  and  junior  enlisted  personnel  and/or  officers,  for 
providing  standardized  training  that  does  not  change  frequently,  for  supporting  hands-on 
training  with  “crawl-phase”  procedural  instruction,  and  as  self-development  training 
following  branch  changes  (e.g.,  armor  to  infantry). 

5.  The  quality  of  generic  content  and  search  findings  will  be  critical  to  the  adoption  of  the 
tool.  Trainers  have  limited  time  to  create  training  and  will  fall  back  on  current 
alternatives  unless  the  TA  can  be  used  very  quickly  and  easily. 

Conclusions  and  Recommendations  for  Future  Research 
Next  Steps  Based  on  Alpha  Testing  Results 
Generic  Base  Content  Development 

Alpha  testing  participants  listed  quality  generic  base  content  as  one  of  the  most  important 
features  of  the  tool.  The  creation  of  accurate,  relevant  content  will  require  the  extensive  support 
of  Army  subject  matter  experts  (SMEs).  The  SMEs  must  have  a  solid  understanding  of  Army 
doctrine,  an  in-depth  knowledge  of  current  enemy  tactics,  techniques,  and  procedures,  and  recent 
deployment  experience.  Ideally,  they  have  themselves  been  a  unit  trainer  so  they  can  represent 
the  needs  of  the  TA  target  audience.  The  base  content  development  process  should  include 
identification  of  the  subset  of  doctrinal  tasks  and  learning  objectives  for  which  base  content  will 
be  written,  collaboration  between  a  SME  and  an  instructional  designer  to  create  the  base  content 
for  each  identified  task  and  LO,  and  review  of  the  content  by  target  users  of  the  TA. 

Search  Feature  Additions 

Alpha  testers  also  indicated  that  the  search  results  must  be  improved  in  order  for  the 
search  to  be  useful.  Improvements  should  be  made  by  expanding  the  search  beyond  the  public 
Internet.  Examples  of  additional  sites  and  databases  that  should  be  searched  include  the  Center 
for  Army  Lessons  Learned,  Anny  Training  and  Evaluation  Program  (ARTEP)  manuals, 
databases  of  post-specific  maps,  and  classified  sites  containing  the  latest  friendly  and  enemy 
TTPs.  While  access  to  classified  information  to  incorporate  into  training  was  identified  as  one  of 
the  most  important  feature  additions  during  the  alpha  testing,  it  is  also  the  most  problematic. 
Once  training  incorporates  classified  information,  it  becomes  classified  itself.  This  means  that 
the  TA  would  need  to  be  able  to  restrict  access  to  training  modules  containing  classified 
information.  Another  potential  issue  with  searching  classified  sites  is  simply  getting  access  and 
pennission  to  search  them.  Finally,  Internet  crawl  results  currently  are  cached  and  pieces  are 
stored  in  the  TA  database.  This  might  need  to  change  if  the  results  include  classified  information. 
In  summary,  a  thorough  study  should  be  done  to  detennine  the  feasibility  and  implications  of 
searching  the  classified  Internet. 
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Course  and  Activity  Template  Enhancements 

Alpha  testing  of  the  current  TA  revealed  a  number  of  usability  enhancements  that  should 
be  made  to  many  of  the  activity  templates.  For  example,  testers  suggested  combining  the  control 
measures  and  military  symbols  tools  in  the  overlay  creation  template  into  a  single,  more  flexible 
tool.  Additionally,  only  three  of  the  six  activity  templates  identified  by  the  Phase  II  research  have 
been  created.  The  three  additional  templates  (e.g.,  situational  judgment,  synchronous 
collaboration,  and  location  selection)  should  be  created  and  incorporated  into  the  tool.  Overall 
course  content  templates  should  also  be  made  more  flexible  and  customizable  to  a  trainer’s 
needs.  Specifically,  users  should  be  able  to  rearrange  pages,  insert  pages,  and  change  the  types  of 
content  found  on  a  page  from  the  default  types  to  a  custom  combination. 

Access  to  the  Tool 

Access  to  the  tool  was  also  a  concern  identified  in  the  alpha  testing.  While  access  to 
computers  cannot  be  controlled,  access  to  the  TA  can  be  ensured  to  any  Soldier  that  has  a 
computer  with  Army  network  access  by  uploading  it  to  an  Army  network.  In  addition  to 
providing  broad  access  to  the  tool,  this  would  allow  trainers  to  begin  using  it  and  adding  to  the 
library  of  base  content.  Each  new  training  module  created  will  make  the  tool  a  richer  and  more 
valuable  resource  for  generating  subsequent  training.  Finally,  access  to  classified  content  may  be 
more  feasible  if  the  TA  is  hosting  on  an  Army  network.  Additionally,  future  efforts  will  work 
towards  developing  a  Sharable  Content  Object  Reference  Model  (SCORM)  compliant  system 
and  link  the  system  to  an  Army  approved  learning  management  system  to  further  enhance  access 
to  the  tool  and  training  materials. 


Summary 

Further  research  and  development  will  help  provide  the  base  content  that  is  crucial  to 
improving  the  speed  with  which  training  can  be  generated,  will  ensure  that  all  relevant  resources 
are  easily  searched  and  accessed,  will  make  the  tool  more  flexible  and  customizable  to  a  trainer’s 
needs,  and  will  ensure  that  the  tool  is  accessible  by  anyone  who  can  connect  to  an  Army 
network.  With  these  enhancements,  the  TA  will  allow  Army  trainers  to  leverage  the  knowledge 
and  experience  of  other  trainers  to  quickly  create  customized,  instructionally  sound  training  that 
meets  their  needs. 
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Appendix  A 
List  of  Acronyms 


AAR 

After  Action  Review 

AC 

Active  Component 

AIT 

Advanced  Individual  Training 

AKO 

Army  Knowledge  Online 

AO 

Area  of  Operations 

ARFORGEN 

Army  Force  Generation 

ARTEP 

Army  Training  and  Evaluation  Program 

BCKS 

Battle  Command  Knowledge  System 

BNCOC 

Basic  Non-Commissioned  Officers’  Course 

CAC 

Common  Access  Card 

CALL 

Center  for  Army  Lessons  Learned 

CARL 

Combined  Arms  Research  Library 

CBT 

Computer  Based  Training 

CO  CDR 

Company  Commander 

CPX 

Command  Post  Exercise 

CTC 

Combat  Training  Center 

DA 

Department  of  the  Army 

Dbase 

Database 

DMX 

Decision-Making  Exercise 

DTF 

Digital  Training  Facility 

EPW 

Enemy  Prisoner  of  War 

FBCB2 

Force  XXI  Battle  Command,  Brigade-and-Below 

FM 

Field  Manual 

FOUO 

For  Official  Use  Only 

FRAGO 

Fragmentary  Orders 

HQ 

Headquarters 

HSOC 

Home  Station  Operations  Center 

HTML 

Hypertext  Markup  Language 

IDF 

Inverse  Document  Frequency 

IE 

Internet  Explorer 

IED 

Improvised  Explosive  Device 

JRTC 

Joint  Readiness  Training  Center 

LFX 

Live  Fire  Exercise 
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LMS 

Learning  Management  System 

LO 

Learning  Objective 

MAPEX 

Map  Exercise 

MDMP 

Military  Decision-Making  Process 

METL 

Mission  Essential  Task  List 

METT-TC 

Mission,  Enemy,  Terrain  and  Weather,  Troops  and  Support,  Time 
Available,  Civil  Considerations 

MSTF 

Mission  Support  Training  Facility 

MOS 

Military  Operational  Specialty 

MS  SQL 

Microsoft  Structured  Query  Language 

NCO 

Non-Commissioned  Officer 

NCSA 

National  Center  for  Supercomputing  Applications 

OBC 

Officer’s  Basic  Course 

OIF 

Operation  Iraqi  Freedom 

OPORD 

Operations  Order 

PL 

Platoon  Leader 

PLDC 

Platoon  Leader’s  Development  Course 

RC 

Reserve  Component 

RDL 

Reimer  Digital  Library 

ROE 

Rules  of  Engagement 

ROI 

Rules  of  Interaction 

RSOI 

Reception,  Staging,  Onward  Movement,  and  Integration 

SCORM 

Sharable  Content  Object  Reference  Model 

SGT 

Sergeant 

SME 

Subject  Matter  Expert 

SOP 

Standard  Operating  Procedure 

SQL 

Structured  Query  Language 

STTR 

Small  Business  Technology  Transfer 

STX 

Situational  Training  Exercise 

TA 

Training  Assistant 

TASS  BN 

Total  Army  School  System  Battalion 

TEWT 

Training/Tactical  Exercise  without  Troops 

IF 

Term  Frequency 

TF-IDF 

Term  Frequency  -  Inverse  Document  Frequency 

TLAC 

Think  Like  a  Commander 

HP 

Tactics,  Techniques,  and  Procedures 

URL 

Uniform  Resource  Locator 

A-2 


WLC  Warrior  Leaders’ Course 

XML  Extensible  Markup  Language 
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Appendix  B 

Background  Tasks  and  Their  Associated  Learning  Objectives 


Task:  Route  Reconnaissance  (or  Reconnaissance  Patrol) _ 

Plan  a  mission  -  Conduct  a  map  reconnaissance/establish  control  measures 

Plan  a  mission  -  Select  mounted  or  dismounted  reconnaissance  based  on  METT-TC 

Plan  a  mission  -  Organize  unit  as  necessary  to  accomplish  mission 

Plan  a  mission  -  Coordinate  with  adjacent  units 

Identify  safe  routes  for  tactical  movement 

Plan  a  mission  -  Address  actions  on  contact  with  the  enemy  or  belligerents 
Plan  a  mission  -  Plan  for  continuous  or  redundant  communications 
Plan  a  mission  -  Coordinate  with  higher  (resources  or  procedures) 

Plan  the  employment  of  fire  support  -  Determine  the  purpose  and  effect  of  indirect  fires 

Plan  the  employment  of  fire  support  -  Identify  priority  targets 

Prepare  for  the  employment  of  fire  support  -  Prepare  observation  plan 

Prepare  for  the  employment  of  fire  support  -  Prepare  trigger  lines 

Visualize,  describe,  and  direct  operations 

Maintain  operations  security  -  Protect  friendly  information 

Adjust  a  plan  based  on  new  intelligence 

Establish  radio  communications  -  Locate  radio  stations 

Take  action  on  contact  -  Make  new  COA  recommendation 

Interact  with  local  nationals 

Work  with  interpreters 

React  to  a  civil  disturbance 

React  to  ambush  -  Locate  threat  forces 

React  to  snipers  -  Identify  sniper  positions 

Treat  and  evacuate  casualties  -  Perfonn  medical  treatment 

Treat  and  evacuate  casualties  -  Support  medical  treatment 

Assume  a  leadership  role 

Handle  enemy  prisoners  of  war  (EPWs)  -  Keep  EPWs  silent 
Handle  enemy  prisoners  of  war  (EPWs)  -  Safeguard  EPWs 
ROE/ROI  application 

Process  captured  documents  and  equipment  -  Tag  all  captured  documents  and  equipment 

Process  captured  documents  and  equipment  -  Evacuate  captured  documents 

Report  tactical  information 

Prepare  a  post-reconnaissance  overlay 

Consolidate  and  reorganize 

Rehearsal  -  Pre-combat  check/Pre-combat  inspection 
Rehearsal  -  Confirmation  brief 
Rehearsal  -  Backbrief 
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Task:  Observation  Post  Operations _ 

Plan  a  mission  -  Conduct  a  map  reconnaissance/establish  control  measures 
Plan  a  mission  -  Organize  unit  as  necessary  to  accomplish  mission 
Identify  safe  routes  for  tactical  movement 
Plan  a  mission  -  Coordinate  with  adjacent  units 

Plan  a  mission  -  Address  actions  on  contact  with  the  enemy  or  belligerents 

Plan  a  mission  -  Detennine  appropriate  manning  level  for  continuous  operation 

Plan  a  mission  -  Identify  special  mission  requirements 

Plan  a  mission  -  Plan  for  continuous  or  redundant  communications 

Plan  the  employment  of  fire  support  -  Determine  the  purpose  and  effect  of  indirect  fires 

Plan  the  employment  of  fire  support  -  Identify  priority  targets 

Prepare  for  the  employment  of  fire  support  -  Prepare  observation  plan 

Prepare  for  the  employment  of  fire  support  -  Prepare  trigger  lines 

Visualize,  describe,  and  direct  operations 

Maintain  operations  security  -  Protect  friendly  information 

Plan  fighting/battle  positions 

Adjust  a  plan  based  on  new  intelligence 

Construct  an  observation  post 

ROE/ROI  application 

Establish  radio  communications  -  Locate  radio  stations 
React  to  ambush  -  Locate  threat  forces 
React  to  snipers  -  Identify  sniper  positions 
Assume  a  leadership  role 

Treat  and  evacuate  casualties  -  Perform  medical  treatment 
Treat  and  evacuate  casualties  -  Support  medical  treatment 
Report  tactical  information 

Rehearsal  -  Pre-combat  check/Pre-combat  inspection 
Rehearsal  -  Confirmation  brief 
Rehearsal  -  Backbrief 
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Task:  Reconnoiter  a  Built-up  Area  or  Urban  Reconnaissance _ 

Plan  a  mission  -  Conduct  a  map  reconnaissance/establish  control  measures 

Plan  a  mission  -  Select  mounted  or  dismounted  reconnaissance  based  on  METT-TC 

Plan  a  mission  -  Organize  unit  as  necessary  to  accomplish  mission 

Identify  safe  routes  for  tactical  movement 

Plan  a  mission  -  Coordinate  with  adjacent  units 

Plan  a  mission  -  Plan  for  continuous  or  redundant  communications 

Plan  a  mission  -  Identify  enemy  safe  havens 

Plan  a  mission  -  Identify  hazardous  and/or  danger  areas 

Plan  a  mission  -  Address  actions  on  contact  with  the  enemy  or  belligerents 

Plan  a  mission  -  Coordinate  with  higher  (resources  or  procedures) 

Plan  the  employment  of  fire  support  -  Determine  the  purpose  and  effect  of  indirect  fires 

Plan  the  employment  of  fire  support  -  Identify  priority  targets 

Prepare  for  the  employment  of  fire  support  -  Prepare  observation  plan 

Prepare  for  the  employment  of  fire  support  -  Prepare  trigger  lines 

Visualize,  describe,  and  direct  operations 

Maintain  operations  security  -  Protect  friendly  information 

Adjust  a  plan  based  on  new  intelligence 

Establish  radio  communications  -  Locate  radio  stations 

React  to  ambush  -  Locate  threat  forces 

React  to  snipers  -  Identify  sniper  positions 

Take  action  on  contact  -  Make  new  COA  recommendation 

Assume  a  leadership  role 

Treat  and  evacuate  casualties  -  Perfonn  medical  treatment 
Treat  and  evacuate  casualties  -  Support  medical  treatment 
Interact  with  local  nationals 
Work  with  interpreters 

Handle  enemy  prisoners  of  war  (EPWs)  -  Keep  EPWs  silent 
Handle  enemy  prisoners  of  war  (EPWs)  -  Safeguard  EPWs 
ROE/ROI  application 

Process  captured  documents  and  equipment  -  Tag  all  captured  documents  and  equipment 

Process  captured  documents  and  equipment  -  Evacuate  captured  documents 

Report  tactical  information 

Prepare  a  post-reconnaissance  overlay 

Consolidate  and  reorganize 

Rehearsal  -  Pre-combat  check/Pre-combat  inspection 
Rehearsal  -  Confirmation  brief 
Rehearsal  -  Backbrief 
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Task:  Building  Assault _ 

Plan  a  mission  -  Conduct  a  map  reconnaissance/establish  control  measures 

Plan  a  mission  -  Organize  unit  as  necessary  to  accomplish  mission 

Plan  a  mission  -  Identify  major  and/or  key  terrain  features 

Identify  safe  routes  for  tactical  movement 

Plan  a  mission  -  Coordinate  with  adjacent  units 

Plan  a  mission  -  Plan  for  continuous  or  redundant  communications 

Plan  a  mission  -  Coordinate  with  higher  (resources  or  procedures) 

Plan  the  employment  of  fire  support  -  Determine  the  purpose  and  effect  of  indirect  fires 

Plan  the  employment  of  fire  support  -  Identify  priority  targets 

Prepare  for  the  employment  of  fire  support  -  Prepare  observation  plan 

Prepare  for  the  employment  of  fire  support  -  Prepare  trigger  lines 

Visualize,  describe,  and  direct  operations 

Maintain  operations  security  -  Protect  friendly  information 

Adjust  a  plan  based  on  new  intelligence 

Establish  radio  communications  -  Locate  radio  stations 

Take  action  on  contact  -  Make  new  COA  recommendation 

Treat  and  evacuate  casualties  -  Perfonn  medical  treatment 

Treat  and  evacuate  casualties  -  Support  medical  treatment 

Assume  a  leadership  role 

Handle  enemy  prisoners  of  war  (EPWs)  -  Keep  EPWs  silent 
Handle  enemy  prisoners  of  war  (EPWs)  -  Safeguard  EPWs 
Secure  uncooperative  civilians  during  operations 
ROE/ROI  application 

Process  captured  documents  and  equipment  -  Tag  all  captured  documents  and  equipment 

Process  captured  documents  and  equipment  -  Evacuate  captured  documents 

Report  tactical  information 

Manage  unintended  consequences 

Consolidate  and  reorganize 

Rehearsal  -  Pre-combat  check/Pre-combat  inspection 
Rehearsal  -  Confirmation  brief 
Rehearsal  -  Backbrief 


B-4 


Task:  Presence  Patrol _ 

Plan  a  mission  -  Conduct  a  map  reconnaissance/establish  control  measures 
Plan  a  mission  -  Organize  unit  as  necessary  to  accomplish  mission 
Plan  a  mission  -  Coordinate  with  adjacent  units 

Plan  a  mission  -  Address  actions  on  contact  with  the  enemy  or  belligerents 

Plan  a  mission  -  Identify  special  mission  requirements 

Plan  a  mission  -  Plan  for  continuous  or  redundant  communications 

Plan  a  mission  -  Identify  hazardous  and/or  danger  areas 

Plan  a  mission  -  Coordinate  with  higher  (resources  or  procedures) 

Plan  a  mission  -  Plan  for  unit  security 

Plan  a  mission  -  Obtain  information  about  persons  to  be  apprehended 

Plan  a  mission  -  Select  tactical  movement  techniques  and  formation  options 

Visualize,  describe,  and  direct  operations 

Maintain  operations  security  -  Protect  friendly  information 

Adjust  a  plan  based  on  new  intelligence 

Coordinate  with  reserve  force 

Interact  with  local  nationals 

Work  with  interpreters 

React  to  a  civil  disturbance 

React  to  ambush  -  Locate  threat  forces 

React  to  snipers  -  Identify  sniper  positions 

Take  action  on  contact  -  Make  new  COA  recommendation 

Treat  and  evacuate  casualties  -  Perfonn  medical  treatment 

Treat  and  evacuate  casualties  -  Support  medical  treatment 

Assume  a  leadership  role 

Handle  enemy  prisoners  of  war  (EPWs)  -  Keep  EPWs  silent 
Handle  enemy  prisoners  of  war  (EPWs)  -  Safeguard  EPWs 
Secure  uncooperative  civilians  during  operations 
ROE/ROI  application 

Process  captured  documents  and  equipment  -  Tag  all  captured  documents  and  equipment 
Process  captured  documents  and  equipment  -  Evacuate  captured  documents 
Report  tactical  information 

Rehearsal  -  Pre-combat  check/Pre-combat  inspection 
Rehearsal  -  Confirmation  brief 
Rehearsal  -  Backbrief 


B-5 


Task:  Conduct  a  Raid  or  Combat  Patrol _ 

Plan  a  mission  -  Conduct  a  map  reconnaissance/establish  control  measures 

Plan  a  mission  -  Organize  unit  as  necessary  to  accomplish  mission 

Identify  safe  routes  for  tactical  movement 

Assign  positions  for  raid  elements 

Plan  a  mission  -  Coordinate  with  adjacent  units 

Plan  a  mission  -  Plan  for  continuous  or  redundant  communications 

Plan  a  mission  -  Identify  major  and/or  key  terrain  features 

Plan  a  mission  -  Coordinate  with  higher  (resources  or  procedures) 

Plan  the  employment  of  fire  support  -  Determine  the  purpose  and  effect  of  indirect  fires 

Plan  the  employment  of  fire  support  -  Identify  priority  targets 

Prepare  for  the  employment  of  fire  support  -  Prepare  observation  plan 

Prepare  for  the  employment  of  fire  support  -  Prepare  trigger  lines 

Visualize,  describe,  and  direct  operations 

Maintain  operations  security  -  Protect  friendly  information 

Adjust  a  plan  based  on  new  intelligence 

Set  the  conditions  for  a  raid 

Establish  radio  communications  -  Locate  radio  stations 
Treat  and  evacuate  casualties  -  Perfonn  medical  treatment 
Treat  and  evacuate  casualties  -  Support  medical  treatment 
Assume  a  leadership  role 

Handle  enemy  prisoners  of  war  (EPWs)  -  Keep  EPWs  silent 
Handle  enemy  prisoners  of  war  (EPWs)  -  Safeguard  EPWs 
ROE/ROI  application 

Process  captured  documents  and  equipment  -  Tag  all  captured  documents  and  equipment 

Process  captured  documents  and  equipment  -  Evacuate  captured  documents 

Report  tactical  information 

Manage  unintended  consequences 

Consolidate  and  reorganize 

Rehearsal  -  Pre-combat  check/Pre-combat  inspection 
Rehearsal  -  Confirmation  brief 
Rehearsal  -  Backbrief 


B-6 


Task:  Security  Patrol _ 

Plan  a  mission  -  Conduct  a  map  reconnaissance/establish  control  measures 
Plan  a  mission  -  Organize  unit  as  necessary  to  accomplish  mission 
Plan  a  mission  -  Coordinate  with  adjacent  units 

Plan  a  mission  -  Address  actions  on  contact  with  the  enemy  or  belligerents 

Plan  a  mission  -  Identify  special  mission  requirements 

Plan  a  mission  -  Plan  for  continuous  or  redundant  communications 

Plan  a  mission  -  Identify  hazardous  and/or  danger  areas 

Plan  a  mission  -  Plan  for  unit  security 

Plan  a  mission  -  Assign  additional  tasks  as  required  to  conduct  the  mission 
Plan  a  mission  -  Coordinate  with  higher  (resources  or  procedures) 

Plan  the  employment  of  fire  support  -  Determine  the  purpose  and  effect  of  indirect  fires 

Plan  the  employment  of  fire  support  -  Identify  priority  targets 

Prepare  for  the  employment  of  fire  support  -  Prepare  observation  plan 

Prepare  for  the  employment  of  fire  support  -  Prepare  trigger  lines 

Visualize,  describe,  and  direct  operations 

Maintain  operations  security  -  Protect  friendly  information 

Adjust  a  plan  based  on  new  intelligence 

React  to  a  civil  disturbance 

React  to  ambush  -  Locate  threat  forces 

React  to  snipers  -  Identify  sniper  positions 

Treat  and  evacuate  casualties  -  Perfonn  medical  treatment 

Treat  and  evacuate  casualties  -  Support  medical  treatment 

Assume  a  leadership  role 

Handle  enemy  prisoners  of  war  (EPWs)  -  Keep  EPWs  silent 
Handle  enemy  prisoners  of  war  (EPWs)  -  Safeguard  EPWs 
ROE/ROI  application 

Process  captured  documents  and  equipment  -  Tag  all  captured  documents  and  equipment 

Process  captured  documents  and  equipment  -  Evacuate  captured  documents 

Report  tactical  information 

Manage  unintended  consequences 

Consolidate  and  reorganize 

Rehearsal  -  Pre-combat  check/Pre-combat  inspection 
Rehearsal  -  Confirmation  brief 
Rehearsal  -  Backbrief 


B-7 


Task:  Convoy  Escort _ 

Plan  a  mission  -  Conduct  a  map  reconnaissance/establish  control  measures 
Plan  a  mission  -  Organize  unit  as  necessary  to  accomplish  mission 
Plan  a  mission  -  Coordinate  with  adjacent  units 

Plan  a  mission  -  Address  actions  on  contact  with  the  enemy  or  belligerents 
Plan  a  mission  -  Plan  for  continuous  or  redundant  communications 
Plan  a  mission  -  Identify  hazardous  and/or  danger  areas 
Plan  a  mission  -  Coordinate  with  higher  (resources  or  procedures) 

Plan  a  mission  -  Detennine  location  of  unit  in  relation  to  convoy 

Plan  a  mission  -  Determine  actions  at  halts 

Plan  a  mission  -  Detennine  actions  in  case  of  vehicle  breakdown 

Plan  a  mission  -  Detennine  actions  in  built-up  areas 

Plan  a  mission  -  Detennine  riot  control  procedures 

Visualize,  describe,  and  direct  operations 

Maintain  operations  security  -  Protect  friendly  information 

Establish  radio  communications  -  Locate  radio  stations 

Adjust  a  plan  based  on  new  intelligence 

Make  recommendations  for  dealing  with  obstacles 

Take  action  on  contact  -  Make  new  COA  recommendation 

React  to  a  civil  disturbance 

React  to  ambush  -  Locate  threat  forces 

React  to  snipers  -  Identify  sniper  positions 

Assume  a  leadership  role 

Treat  and  evacuate  casualties  -  Perform  medical  treatment 
Treat  and  evacuate  casualties  -  Support  medical  treatment 
Handle  enemy  prisoners  of  war  (EPWs)  -  Keep  EPWs  silent 
Handle  enemy  prisoners  of  war  (EPWs)  -  Safeguard  EPWs 
ROE/ROI  application 
Report  tactical  information 
Manage  unintended  consequences 
Consolidate  and  reorganize 

Rehearsal  -  Pre-combat  check/Pre-combat  inspection 
Rehearsal  -  Confirmation  brief 
Rehearsal  -  Backbrief 
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Task:  Checkpoint/Traffic  control  point  operations/Traffic  Control  Point  Operations 

Plan  a  mission  -  Conduct  a  map  reconnaissance/establish  control  measures 
Plan  a  mission  -  Organize  unit  as  necessary  to  accomplish  mission 
Plan  a  mission  -  Coordinate  with  adjacent  units 

Plan  a  mission  -  Address  actions  on  contact  with  the  enemy  or  belligerents 
Plan  a  mission  -  Coordinate  with  higher  (resources  or  procedures) 

Visualize,  describe,  and  direct  operations 

Maintain  operations  security  -  Protect  friendly  information 

Adjust  a  plan  based  on  new  intelligence 

Construct  a  checkpoint/traffic  control  point 

Plan  fighting/battle  positions 

Treat  and  evacuate  casualties  -  Perform  medical  treatment 
Treat  and  evacuate  casualties  -  Support  medical  treatment 
Assume  a  leadership  role 

Conduct  a  vehicle  search  with  uncooperative  civilians 
Work  with  interpreters 
Identify  contraband  or  prohibited  items 
ROE/ROI  application 

Handle  enemy  prisoners  of  war  (EPWs)  -  Keep  EPWs  silent 
Handle  enemy  prisoners  of  war  (EPWs)  -  Safeguard  EPWs 

Process  captured  documents  and  equipment  -  Tag  all  captured  documents  and  equipment 

Process  captured  documents  and  equipment  -  Evacuate  captured  documents 

Report  tactical  information 

Secure  uncooperative  civilians  during  operations 

Manage  unintended  consequences 

Consolidate  and  reorganize 

Rehearsal  -  Pre-combat  check/Pre-combat  inspection 
Rehearsal  -  Confirmation  brief 
Rehearsal  -  Backbrief 
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Task:  Building  Search _ 

Plan  a  mission  -  Conduct  a  map  reconnaissance/establish  control  measures 

Plan  a  mission  -  Organize  unit  as  necessary  to  accomplish  mission 

Plan  a  mission  -  Coordinate  with  adjacent  units 

Plan  a  mission  -  Plan  for  continuous  or  redundant  communications 

Plan  a  mission  -  Identify  major  and/or  key  terrain  features 

Plan  a  mission  -  Coordinate  with  higher  (resources  or  procedures) 

Plan  a  mission  -  Plan  contingencies  for  obstacles  and  danger  areas 
Plan  a  mission  -  Select  method  to  control  building  occupants 
Identify  safe  routes  for  tactical  movement 

Plan  the  employment  of  fire  support  -  Determine  the  purpose  and  effect  of  indirect  fires 

Plan  the  employment  of  fire  support  -  Identify  priority  targets 

Prepare  for  the  employment  of  fire  support  -  Prepare  observation  plan 

Prepare  for  the  employment  of  fire  support  -  Prepare  trigger  lines 

Visualize,  describe,  and  direct  operations 

Maintain  operations  security  -  Protect  friendly  information 

Adjust  a  plan  based  on  new  intelligence 

Establish  radio  communications  -  Locate  radio  stations 

Work  with  interpreters 

Take  action  on  contact  -  Make  new  COA  recommendation 
Treat  and  evacuate  casualties  -  Perfonn  medical  treatment 
Treat  and  evacuate  casualties  -  Support  medical  treatment 
Assume  a  leadership  role 

Handle  enemy  prisoners  of  war  (EPWs)  -  Keep  EPWs  silent 
Handle  enemy  prisoners  of  war  (EPWs)  -  Safeguard  EPWs 
Secure  uncooperative  civilians  during  operations 
ROE/ROI  application 

Process  captured  documents  and  equipment  -  Tag  all  captured  documents  and  equipment 

Process  captured  documents  and  equipment  -  Evacuate  captured  documents 

Report  tactical  information 

Manage  unintended  consequences 

Consolidate  and  reorganize 

Rehearsal  -  Pre-combat  check/Pre-combat  inspection 
Rehearsal  -  Confirmation  brief 
Rehearsal  -  Backbrief 
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Task:  Route  Security  (or  IED  Patrol) _ 

Plan  a  mission  -  Conduct  a  map  reconnaissance/establish  control  measures 
Plan  a  mission  -  Organize  unit  as  necessary  to  accomplish  mission 
Plan  a  mission  -  Coordinate  with  adjacent  units 

Plan  a  mission  -  Address  actions  on  contact  with  the  enemy  or  belligerents 
Plan  a  mission  -  Plan  for  continuous  or  redundant  communications 
Plan  a  mission  -  Identify  hazardous  and/or  danger  areas 
Plan  a  mission  -  Coordinate  with  higher  (resources  or  procedures) 

Plan  the  employment  of  fire  support  -  Determine  the  purpose  and  effect  of  indirect  fires 

Plan  the  employment  of  fire  support  -  Identify  priority  targets 

Prepare  for  the  employment  of  fire  support  -  Prepare  observation  plan 

Prepare  for  the  employment  of  fire  support  -  Prepare  trigger  lines 

Visualize,  describe,  and  direct  operations 

Maintain  operations  security  -  Protect  friendly  information 

Interact  with  local  nationals 

Adjust  a  plan  based  on  new  intelligence 

Establish  radio  communications  -  Locate  radio  stations 

Coordinate  with  reserve  force 

Take  action  on  contact  -  Make  new  COA  recommendation 

React  to  ambush  -  Locate  threat  forces 

React  to  snipers  -  Identify  sniper  positions 

Treat  and  evacuate  casualties  -  Perfonn  medical  treatment 

Treat  and  evacuate  casualties  -  Support  medical  treatment 

Assume  a  leadership  role 

Handle  enemy  prisoners  of  war  (EPWs)  -  Keep  EPWs  silent 
Handle  enemy  prisoners  of  war  (EPWs)  -  Safeguard  EPWs 
ROE/ROI  application 

Process  captured  documents  and  equipment  -  Tag  all  captured  documents  and  equipment 
Process  captured  documents  and  equipment  -  Evacuate  captured  documents 
Report  tactical  information 
Consolidate  and  reorganize 

Rehearsal  -  Pre-combat  check/Pre-combat  inspection 
Rehearsal  -  Confirmation  brief 
Rehearsal  -  Backbrief 
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Task:  Cordon  and  knock/Cordon  and  search _ 

Plan  a  mission  -  Conduct  a  map  reconnaissance/establish  control  measures 
Plan  a  mission  -  Organize  unit  as  necessary  to  accomplish  mission 
Plan  a  mission  -  Coordinate  with  adjacent  units 

Plan  a  mission  -  Address  actions  on  contact  with  the  enemy  or  belligerents 
Plan  a  mission  -  Plan  for  continuous  or  redundant  communications 
Plan  a  mission  -  Identify  hazardous  and/or  danger  areas 
Plan  a  mission  -  Coordinate  with  higher  (resources  or  procedures) 

Plan  the  employment  of  fire  support  -  Determine  the  purpose  and  effect  of  indirect  fires 

Plan  the  employment  of  fire  support  -  Identify  priority  targets 

Prepare  for  the  employment  of  fire  support  -  Prepare  observation  plan 

Prepare  for  the  employment  of  fire  support  -  Prepare  trigger  lines 

Maintain  operations  security  -  Protect  friendly  information 

Adjust  a  plan  based  on  new  intelligence 

Visualize,  describe,  and  direct  operations 

Identify  safe  routes  for  tactical  movement 

Take  action  on  contact  -  Make  new  COA  recommendation 

Interact  with  local  nationals 

Work  with  interpreters 

React  to  snipers  -  Identify  sniper  positions 

Treat  and  evacuate  casualties  -  Perfonn  medical  treatment 

Treat  and  evacuate  casualties  -  Support  medical  treatment 

Assume  a  leadership  role 

Handle  enemy  prisoners  of  war  (EPWs)  -  Keep  EPWs  silent 
Handle  enemy  prisoners  of  war  (EPWs)  -  Safeguard  EPWs 
ROE/ROI  application 

Process  captured  documents  and  equipment  -  Tag  all  captured  documents  and  equipment 

Process  captured  documents  and  equipment  -  Evacuate  captured  documents 

Report  tactical  information 

Manage  unintended  consequences 

Consolidate  and  reorganize 

Rehearsal  -  Pre-combat  check/Pre-combat  inspection 
Rehearsal  -  Confirmation  brief 
Rehearsal  -  Backbrief 
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Task:  Conduct  an  Attack _ 

Plan  a  mission  -  Conduct  a  map  reconnaissance/establish  control  measures 
Plan  a  mission  -  Organize  unit  as  necessary  to  accomplish  mission 
Plan  a  mission  -  Coordinate  with  adjacent  units 

Plan  a  mission  -  Address  actions  on  contact  with  the  enemy  or  belligerents 
Plan  a  mission  -  Plan  for  continuous  or  redundant  communications 
Plan  a  mission  -  Identify  major  and/or  key  terrain  features 
Plan  a  mission  -  Coordinate  with  higher  (resources  or  procedures) 

Plan  the  employment  of  fire  support  -  Determine  the  purpose  and  effect  of  indirect  fires 

Plan  the  employment  of  fire  support  -  Identify  priority  targets 

Prepare  for  the  employment  of  fire  support  -  Prepare  observation  plan 

Prepare  for  the  employment  of  fire  support  -  Prepare  trigger  lines 

Visualize,  describe,  and  direct  operations 

Identify  safe  routes  for  tactical  movement 

Maintain  operations  security  -  Protect  friendly  information 

Adjust  a  plan  based  on  new  intelligence 

Establish  radio  communications  -  Locate  radio  stations 

Take  action  on  contact  -  Make  new  COA  recommendation 

Treat  and  evacuate  casualties  -  Perfonn  medical  treatment 

Treat  and  evacuate  casualties  -  Support  medical  treatment 

Assume  a  leadership  role 

Handle  enemy  prisoners  of  war  (EPWs)  -  Keep  EPWs  silent 
Handle  enemy  prisoners  of  war  (EPWs)  -  Safeguard  EPWs 
ROE/ROI  application 

Process  captured  documents  and  equipment  -  Tag  all  captured  documents  and  equipment 
Process  captured  documents  and  equipment  -  Evacuate  captured  documents 
Report  tactical  information 
Consolidate  and  reorganize 

Rehearsal  -  Pre-combat  check/Pre-combat  inspection 
Rehearsal  -  Confirmation  brief 
Rehearsal  -  Backbrief 
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Task:  Conduct  a  Defense _ 

Plan  a  mission  -  Conduct  a  map  reconnaissance/establish  control  measures 
Plan  a  mission  -  Organize  unit  as  necessary  to  accomplish  mission 
Plan  a  mission  -  Coordinate  with  adjacent  units 

Plan  a  mission  -  Address  actions  on  contact  with  the  enemy  or  belligerents 
Plan  a  mission  -  Plan  for  continuous  or  redundant  communications 
Plan  a  mission  -  Develop  an  integrated  obstacle  plan 

Plan  the  employment  of  fire  support  -  Determine  the  purpose  and  effect  of  indirect  fires 

Plan  the  employment  of  fire  support  -  Identify  priority  targets 

Prepare  for  the  employment  of  fire  support  -  Prepare  observation  plan 

Prepare  for  the  employment  of  fire  support  -  Prepare  trigger  lines 

Plan  a  mission  -  Develop  disengagement  criteria 

Identify  safe  routes  for  tactical  movement 

Visualize,  describe,  and  direct  operations 

Maintain  operations  security  -  Protect  friendly  information 

Adjust  a  plan  based  on  new  intelligence 

Plan  fighting/battle  positions 

Take  action  on  contact  -  Make  new  COA  recommendation 
Assume  a  leadership  role 

Treat  and  evacuate  casualties  -  Perform  medical  treatment 
Treat  and  evacuate  casualties  -  Support  medical  treatment 
Handle  enemy  prisoners  of  war  (EPWs)  -  Keep  EPWs  silent 
Handle  enemy  prisoners  of  war  (EPWs)  -  Safeguard  EPWs 
ROE/ROI  application 

Process  captured  documents  and  equipment  -  Tag  all  captured  documents  and  equipment 
Process  captured  documents  and  equipment  -  Evacuate  captured  documents 
Report  tactical  information 
Consolidate  and  reorganize 

Rehearsal  -  Pre-combat  check/Pre-combat  inspection 
Rehearsal  -  Confirmation  brief 
Rehearsal  -  Backbrief 
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Task:  Movement  to  Contact _ 

Plan  a  mission  -  Conduct  a  map  reconnaissance/establish  control  measures 

Plan  a  mission  -  Organize  unit  as  necessary  to  accomplish  mission 

Plan  a  mission  -  Coordinate  with  adjacent  units 

Plan  a  mission  -  Plan  for  continuous  or  redundant  communications 

Plan  a  mission  -  Select  a  technique  for  conducting  a  movement  to  contact 

Plan  a  mission  -  Coordinate  with  higher  (resources  or  procedures) 

Plan  a  mission  -  Address  actions  on  contact  with  the  enemy  or  belligerents 

Plan  the  employment  of  fire  support  -  Determine  the  purpose  and  effect  of  indirect  fires 

Plan  the  employment  of  fire  support  -  Identify  priority  targets 

Prepare  for  the  employment  of  fire  support  -  Prepare  observation  plan 

Prepare  for  the  employment  of  fire  support  -  Prepare  trigger  lines 

Visualize,  describe,  and  direct  operations 

Maintain  operations  security  -  Protect  friendly  information 

Adjust  a  plan  based  on  new  intelligence 

Treat  and  evacuate  casualties  -  Perform  medical  treatment 

Treat  and  evacuate  casualties  -  Support  medical  treatment 

Assume  a  leadership  role 

React  to  snipers  -  Identify  sniper  positions 

React  to  ambush  -  Locate  threat  forces 

Handle  enemy  prisoners  of  war  (EPWs)  -  Keep  EPWs  silent 

Handle  enemy  prisoners  of  war  (EPWs)  -  Safeguard  EPWs 

ROE/ROI  application 

Process  captured  documents  and  equipment  -  Tag  all  captured  documents  and  equipment 
Process  captured  documents  and  equipment  -  Evacuate  captured  documents 
Report  tactical  information 
Consolidate  and  reorganize 

Rehearsal  -  Pre-combat  check/Pre-combat  inspection 
Rehearsal  -  Confirmation  brief 
Rehearsal  -  Backbrief 
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Appendix  C 

General  Training  Need  Categories  and  Their  Associated  Learning  Objectives 


Category:  Visual  Communications  Skills _ 

Learning  objectives  related  to  visual  communications  skills  enhance  trainees’  ability  to  communicate 
mission  plans  or  tactics  using  operational  graphics;  These  skills  are  somewhat  independent  of  digital 
display  use,  although  they  enhance  such  use  because  trainees  will  know  what  it  is  they  wish  to  create  with 

drawing  tools,  etc. _ 

Plan  a  mission  -  Conduct  a  map  reconnaissance/establish  control  measures 
Identify  hazardous  and/or  danger  areas 
Determine  location  of  unit  in  relation  to  convoy 
Identify  safe  routes  for  tactical  movement 

Prepare  the  employment  of  fire  support  -  Prepare  observation  plan 

Prepare  the  employment  of  fire  support  -  Prepare  trigger  lines 

React  to  ambush  -  Locate  threat  forces 

Prepare  a  post-reconnaissance  overlay 

Take  action  on  contact  -  Make  new  COA  recommendation 

Adjust  a  plan  based  on  new  intelligence 

Assign  positions  for  raid  elements 

Plan  a  mission  -  Develop  an  integrated  obstacle  plan 

Category:  Coordination  and  Synchronization _ 

Learning  objectives  related  to  coordination  and  synchronization  enhance  trainees’  ability  to  consider  the 
larger  picture  involved  in  mission  planning  and  execution;  That  larger  picture  may  include  adjacent, 

supporting,  higher,  or  reserve  units  and/or  assets _ 

Plan  a  mission  -  Conduct  a  map  reconnaissance/establish  control  measures 
Plan  a  mission  -  Coordinate  with  adjacent  units 

Plan  a  mission  -  Determine  appropriate  manning  level  for  continuous  operation 
Plan  a  mission  -  Plan  for  continuous  or  redundant  communications 
Plan  a  mission  -  Coordinate  with  higher  (resources  or  procedures) 

Plan  a  mission  -  Determine  location  of  unit  in  relation  to  convoy 

Plan  the  employment  of  fire  support  -  Determine  the  purpose  and  effect  of  indirect  fires 

Prepare  for  the  employment  of  fire  support  -  Prepare  observation  plan 

Prepare  for  the  employment  of  fire  support  -  Prepare  trigger  lines 

Coordinate  with  adjacent  elements 

Coordinate  with  reserve  force 

Set  the  conditions  for  a  raid 

Plan  a  mission  -  Develop  an  integrated  obstacle  plan 

Consolidate  and  reorganize _ 

Category:  Tactical  Decision-making _ 

Learning  objectives  related  to  tactical  decision-making  enhance  trainees’  ability  to  make  tactically  sound 

decisions  regarding  the  organization  and  employment  of  troops  and  combat  support _ 

Plan  a  mission  -  Organize  unit  as  necessary  to  accomplish  mission 

Plan  a  mission  -  Select  mounted  or  dismounted  reconnaissance  based  on  METT-TC 

Plan  a  mission  -  Determine  appropriate  manning  level  for  continuous  operation 

Plan  a  mission  -  Select  method  to  control  building  occupants 

Plan  a  mission  -  Select  tactical  movement  techniques  and  formation  options 

Plan  a  mission  -  Select  a  technique  for  conducting  a  movement  to  contact 

Visualize,  describe,  and  direct  operations 

Plan  the  employment  of  fire  support  -  Determine  the  purpose  and  effect  of  indirect  fires _ 
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Plan  a  mission  -  Develop  disengagement  criteria 

Take  action  on  contact  -  Make  new  COA  recommendation 

Adjust  a  plan  based  on  new  intelligence 

Make  recommendations  for  dealing  with  obstacles 

Conduct  actions  at  danger  areas  -  Select  movement  techniques 

Assign  positions  for  raid  elements 

Category:  Rapid  Reaction 

Learning  objectives  related  to  rapid  reaction  enhance  trainees’  ability  to  respond  automatically  to  certain 
combat  events;  Given  the  medium  of  the  TA  tasks  (IMI),  these  learning  objectives  focus  more  on  the 
perceptual  skills  that  facilitate  automatic  responding,  as  opposed  to  the  motor  skills 

React  to  ambush  -  Locate  threat  forces 

React  to  snipers  -  Identify  sniper  positions 

Category:  Communications  Skills 

Learning  objectives  related  to  communications  skills  enhance  trainees’  ability  to  maintain  radio  and  other 
connections  throughout  a  mission;  For  actual  units,  exercises  with  these  learning  objectives  also  may 
facilitate  internalization  of  these  connections;  The  TA  does  not  address  buttonology  skills  for  digital 
equipment,  which  is  handled  by  very  complex  and  expensive  simulations 

Plan  a  mission  -  Plan  for  continuous  or  redundant  communications 

Establish  radio  communications  -  Locate  radio  stations 

Category:  Tactical  Perception 

Learning  objectives  related  to  tactical  perception  enhance  trainees’  sensitivity  to  METT-TC  (especially 
terrain)  factors  that  influence  mission  planning  and  execution;  IMI  is  especially  well  suited  to  enhancing 
tactical  perception  skills 

Plan  a  mission  -  Identify  enemy  safe  havens 

Plan  a  mission  -  Identify  hazardous  and/or  danger  areas 

Plan  a  mission  -  Identify  major  and/or  key  terrain  features 

Plan  a  mission  -  Obtain  information  about  persons  to  be  apprehended 

Identify  safe  routes  for  tactical  movement 

Plan  the  employment  of  fire  support  -  Identify  priority  targets 

Category:  Adaptive  Thinking  and  Contingency  Planning 

Learning  objectives  related  to  adaptive  thinking  and  contingency  planning  enhance  trainees’  ability  to 
consider  unexpected  events  that  may  come  up  during  a  mission;  They  also  enhance  trainees’  flexibility  of 
thinking  in  response  to  such  events 

Plan  a  mission  -  Identify  special  mission  requirements 

Plan  a  mission  -  Plan  for  unit  security 

Plan  a  mission  -  Assign  additional  tasks  as  required  to  conduct  the  mission 

Plan  a  mission  -  Plan  contingencies  for  obstacles  and  danger  areas 

Plan  a  mission  -  Determine  actions  at  halts 

Plan  a  mission  -  Determine  actions  in  case  of  vehicle  breakdown 

Plan  a  mission  -  Determine  actions  in  built-up  areas 

Plan  a  mission  -  Determine  riot  control  procedures 

Adjust  a  plan  based  on  new  intelligence 

Assume  a  leadership  role 

Take  action  on  contact  -  Make  new  COA  recommendation 
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Category:  Interpersonal  Competency 

Learning  objectives  related  to  interpersonal  competency  enhance  trainees’  ability  to  handle  challenging 
situations  involving  other  people,  including  superiors,  subordinates,  members  of  the  sister  services  or  host 
nation  security  forces,  and  host  nation  civilians;  To  some  extent  these  learning  objectives  can  address 
cultural  awareness 

Plan  a  mission  -  Address  actions  on  contact  with  the  enemy  or  belligerents 

Plan  a  mission  -  Select  method  to  control  building  occupants 

Plan  a  mission  -  Determine  riot  control  procedures 

Maintain  operations  security  -  Protect  friendly  information 

Handle  enemy  prisoners  of  war  (EPWs)  -  Search  EPWs 

Handle  enemy  prisoners  of  war  (EPWs)  -  Keep  EPWs  silent 

React  to  civil  disturbance 

Secure  uncooperative  civilians  during  operations 

Conduct  a  vehicle  search  with  uncooperative  civilians 

Work  with  interpreters 

Manage  unintended  consequences 

Interact  with  local  nationals 

Category:  Information  Management 

Learning  objectives  relating  to  information  management  enhance  trainees’  understanding  of  the 
information  networks  in  a  unit,  who  must  know  what  and  when;  For  actual  units,  these  learning  objectives 
may  assist  in  internalizing  such  networks 

Process  captured  documents  and  equipment  -  Tag  all  captured  documents  and  equipment 

Process  captured  documents  and  equipment  -  Evacuate  captured  documents 

Report  tactical  information 

Category:  Procedural  Skills 

Learning  objectives  related  to  procedural  skills  may  vary  widely,  but  have  in  common  their  impact  on 
trainee’s  ability  to  follow  fairly  straightforward  procedures;  The  procedures  addressed  by  these  learning 
objectives  do  not  include  operational  procedures  (e.g.,  equipment  and  vehicles),  although  they  could 

Construct  an  observation  post 

Construct  a  checkpoint/traffic  control  point 

Plan  fighting/battle  positions 

Identify  contraband  or  prohibited  items 

Category:  Medical  Skills 

Learning  objectives  related  to  medical  skills  enhance  trainees’  perceptual  speed  as  it  relates  to  medical 
treatment;  These  “walk-phase”  objectives  are  meant  to  improve  the  speed  with  which  medical  personnel 
make  judgments  although  motor  medical  skills  are  not  addressed 

Treat  and  evacuate  casualties  -  Perform  medical  treatment 

Treat  and  evacuate  casualties  -  Support  medical  treatment 

Category:  ROE/ROI  application 

The  ROE/ROI  learning  objective  enhances  trainees’  ability  to  apply  rules  of  engagement/interaction  to 
making  tough  combat  decisions 

Same  as  category 
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Appendix  D 

Demographic  Survey  Questions 

1 .  Current  Rank: _ 

2.  Current  Duty  position: _ 

3.  Unit  type  (e.g.,  annor,  cavalry,  engineer): _ 

4.  Recent  deployment  experience: 


5.  Please  describe  your  experience  providing  training  for  subordinates  in  your  current  and/or  past 
unit(s). 


Reason  for  Training 
(circle  all  that  apply) 

Teaching  Methods 
(circle  all  that  apply) 

Subject  Taught  (e.g.,  gunnery) 
(please  specify) 

-  Pre-deployment 

-  CTC  rotation 

-  Post-deployment 

-  Refresher/Sustainment 

-  New  personnel 

-  New  equipment 

-  Other  (please  specify): 

-  Briefing/lecture 

-  Computer-based  training 

-  Range  training 

-  Small-group  instruction 

-  Other  (please  specify): 

-  Pre-deployment 

-  CTC  rotation 

-  Post-deployment 

-  Refresher/Sustainment 

-  New  personnel 

-  New  equipment 

-  Other  (please  specify): 

-  Briefing/lecture 

-  Computer-based  training 

-  Range  training 

-  Small-group  instruction 

-  Other  (please  specify): 

-  Pre-deployment 

-  CTC  rotation 

-  Post-deployment 

-  Refresher/Sustainment 

-  New  personnel 

-  New  equipment 

-  Other  (please  specify): 

-  Briefing/lecture 

-  Computer-based  training 

-  Range  training 

-  Small-group  instruction 

-  Other  (please  specify): 

-  Pre-deployment 

-  CTC  rotation 

-  Post-deployment 

-  Refresher/Sustainment 

-  New  personnel 

-  New  equipment 

-  Other  (please  specify): 

-  Briefing/lecture 

-  Computer-based  training 

-  Range  training 

-  Small-group  instruction 

-  Other  (please  specify): 
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6.  Please  indicate  which,  if  any,  of  the  challenges  listed  below  that  you  have  experienced  when 
providing  training  for  subordinates  in  your  current  and/or  past  unit(s).  Circle  all  that  apply. 


o  Technology  limitations  (e.g.,  available  technology  wasn’t  usable  or  didn’t  work) 
o  Personnel  constraints  (e.g.,  no  one  available  to  conduct  perfonnance  assessment) 
o  Resource  limitations  (e.g.,  not  enough  computers,  not  enough  ammunition) 
o  Time  constraints 
o  Difficulty  detennining  what  to  train 
o  Difficulty  detennining  how  to  train 
o  Other: 


7.  Please  list  the  strategies  you  used  to  overcome  these  challenges: 
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Appendix  E 

Fort  Hood  Training  Generation  Protocol 


OPTION#! 

1 .  Be  sure  IE  is  set  to  allow  pop-ups 

2.  Login  to  the  Army  TA  using  your  username  and  password 

3.  Select  “Create  New  Training” 

4.  Select  “Develop  Training  for  a  Doctrinal  Task” 

5.  Select  “Route  Security  (or  IED  Patrol)” 

6.  Select  “Plan  a  Mission  -  Conduct  a  Map  Reconnaissance” 

7.  Rename  Training  to  “(Your  Last  Name)  Route  Security  Training” 

8.  Select  renamed  training 

9.  Edit  the  Introduction  text 

a.  Add  a  hyperlink  to  the  page 

10.  Edit  a  page  in  the  Basic  Tutorial  by  replacing  an  image  with  either  the  “map .jpg”  image 
on  your  desktop  or  an  image  found  via  the  search  (if  available). 

a.  Add  a  caption  to  the  image 

1 1 .  Edit  a  page  in  the  Basic  Tutorial  by  changing  the  size  of  a  photo 

12.  Edit  the  Practical  Exercise  Map  Overlay  Creation  Page  as  follows: 

a.  Insert  a  map  by  selecting  the  designated  image  from  the  desktop 

b.  Edit  the  “base”  overlay  that  students  will  see  as  they  begin  the  practical  exercise. 
When  completed.  Lock  the  overlay  layers  that  you  do  not  want  students  to  edit. 

c.  Edit  the  “correct  answer”  map  overlay  using  the  drawing  tool 

13.  Make  or  suggest  content  edits  as  they  are  noticed 

14.  Copy  completed  training 

15.  Delete  training  copy 

OPTION  #2 

1 .  Be  sure  IE  is  set  to  allow  pop-ups 

2.  Login  to  the  Army  TA  using  your  username  and  password 

3.  Select  “Create  New  Training” 

4.  Select  “Develop  Training  for  a  Doctrinal  Task” 

5.  Select  “Route  Security  (or  IED  Patrol)” 

6.  Select  “Plan  a  Mission  -  Coordinate  with  Adjacent  Units” 

7.  Rename  Training  to  “(Your  Last  Name)  Route  Security  Training” 

8.  Select  renamed  training 

9.  Edit  the  Introduction  text 

a.  Add  a  hyperlink  to  the  page 

10.  Edit  a  page  in  the  Basic  Tutorial  by  replacing  an  image  with  either  the  “map .jpg”  image 
on  your  desktop  or  an  image  found  via  the  search  (if  available). 

a.  Add  a  caption  to  the  image 

1 1 .  Edit  a  page  in  the  Basic  Tutorial  by  changing  the  size  of  a  photo 

12.  In  the  Practical  Exercise,  edit  the  Self-Evaluation  Question  feedback. 

13.  Make  or  suggest  content  edits  as  they  are  noticed 
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14.  Copy  completed  training 

15.  Delete  training  copy 
OPTION  #3 

1 .  Be  sure  IE  is  set  to  allow  pop-ups 

2.  Login  to  the  Army  TA  using  your  username  and  password 

3.  Select  “Create  New  Training” 

4.  Select  “Develop  Training  for  a  Doctrinal  Task” 

5.  Select  “Building  Assault” 

6.  Select  “Plan  a  Mission  -  Determine  the  purpose  and  effect  of  indirect  fires” 

7.  Rename  Training  to  “(Your  Last  Name)  Building  Assault  Training” 

8.  Select  renamed  training 

9.  Edit  the  Introduction  text 

a.  Add  a  hyperlink  to  the  page 

10.  Edit  a  page  in  the  Basic  Tutorial  by  replacing  an  image  with  either  the  “map .jpg”  image 
on  your  desktop  or  an  image  found  via  the  search  (if  available). 

a.  Add  a  caption  to  the  image 

1 1 .  Edit  a  page  in  the  Basic  Tutorial  by  changing  the  size  of  a  photo 

12.  Edit  the  Practical  Exercise  by  modifying  the  question  and  response  options  for  Question 
1 

13.  Make  or  suggest  content  edits  as  they  are  noticed 

14.  Copy  completed  training 

15.  Delete  training  copy 
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Appendix  F 

User  Impressions  Survey 


Usability  (check  the  box  to  indicate  your  answer) 


Very 

Easy 

Easy 

Neither 

Difficult 

Very 

Difficult 

1 .  How  easy  was  it  to  take  an  Army  TA 
training  package? 

2.  How  easy  was  it  to  construct  a  training 
package  using  the  Army  TA? 

3.  How  easy  was  it  to  edit  text? 

4.  How  easy  was  it  to  use  the  search 
function? 

5.  How  easy  was  it  to  insert  images? 

6.  How  easy  was  it  to  use  the  drawing  tool 
(if  applicable)? 

Please  list  any  Army  TA  features  that  posed  difficulty. 


Utility 

1.  How  useful  do  you  think  the  Army  TA  will  be  for  conducting  unit  training  (circle  one)? 
Definitely  Probably  Maybe  Probably  Not  Definitely  Not 

Please  list  the  reasons  why/why  not. 


2.  Could  you  use  the  Anny  TA  to  create  the  kind  of  training  your  unit  needs  (circle  one)? 
Yes  No 

Please  specify  why/why  not. 
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3.  Would  you  use  the  Army  TA  to  create  unit  training  (circle  one)? 

Yes  No 

Please  list  the  challenges  to  using  the  Army  TA  for  conducting  unit  training. 

Please  list  the  feasible  alternatives  to  using  the  Army  TA  for  conducting  unit  training. 


Effectiveness 

How  effective  do  you  think  the  training  generated  by  the  Army  TA  can  be  for  addressing  key 
training  requirements  in  your  unit  (circle  one)? 

Very  Effective  Somewhat  Effective  Neither  Somewhat  Ineffective  Very  Ineffective 
Please  list  the  training  requirements  you  think  the  Anny  TA  training  could  address  effectively. 


General 

1.  What  did  you  particularly  like  about  the  Army  TA? 


2.  What  did  you  particularly  dislike  about  the  Anny  TA? 


3.  What  recommendations  do  you  have  for  improving  the  Anny  TA  (please  list)? 


F-2 


Appendix  G 

Focus  Group  Interview  Protocol 

In  the  focus  groups,  participants  will  be  walked  through  the  screen  shot  booklets  and  asked  to 
describe  their  experiences  with  the  interface  features  and  system  capabilities  represented. 
Emphasis  will  be  placed  on  the  areas  of  observed  difficulty  during  the  testing  sessions. 


For  training  completion,  participants  will  be  asked  about  (screen  captures  designated  in  italics): 

•  Introduction 

•  Basic  Tutorial  (general)  (screen  capture  of  first  page) 

•  Basic  Tutorial  (learning  objective)  (screen  capture  of  first  page) 

•  What  Right  Looks  Like  (screen  capture  of  scenario  page  and  solution  page) 

•  Practical  Exercise  (template  below  depends  on  what  training  module  was  taken) 

o  Map  Overlay  Creation  (screen  capture  of  question  page) 
o  Free-text  (screen  capture  of  question  page) 
o  Test  Question  (screen  capture  of  question  page) 
o  Answer/Feedback  (screen  capture  for  each  type  of  template) 

Questions  will  focus  on  layout,  reading  load,  subject  matter  accuracy  (as  generic  content), 
usability  of  overlay  tool,  utility  of  feedback  (as  generic  content) 


For  training  construction,  participants  will  be  asked  about  (all  items  should  have  a  screen 
capture,  which  is  designated  in  italics): 

•  Editing  text  (screen  capture  of  edit  intro  page  after  edit  has  been  selected  and  the  same 
page  after  the  text  editor  is  showing) 

•  Using  the  search  functionality  (screen  capture  of  initial  search  results  page) 

•  Inserting  images  (screen  capture  of  image  results  page  with  an  image  selected) 

•  Edit  practical  exercise  questions: 

o  Using  the  map  overlay  drawing  tool  (screen  capture  of  the  drawing  tool  in  edit 
mode) 

o  Editing  test  question  text  (screen  capture  of  the  edit  test  question  page) 
o  Editing  self-evaluation  text  (screen  capture  of  the  edit  test  question  page) 

Questions  will  focus  on  usability,  utility  of  generic  content,  and  utility  of  search  engine  over 
alternatives 
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Additional  questions  will  focus  on  interface  suggestions  made  by  users  for  features  that  do  not 
yet  exist.  Questions  will  involve  presenting  users  with  use-case  scenarios  and  paper  prototypes  to 
facilitate  the  discussion. 

Sample  Scenarios  Include: 

1 .  Scenario:  You  are  editing  a  page  that  contains  an  image  and  a  single  block  of  text.  You 
would  like  to  add  a  second  image.  Which  method  would  you  prefer? 

a.  Option  1 :  There  is  a  button  on  the  page  that  says  “Add  Image.”  Clicking  the 
button  allows  you  to  add  an  image  to  the  end  of  the  page. 

b.  Option  2:  There  are  buttons  in  the  text  editor  that  allow  to  you  specify  whether  to 
insert  an  image  before  or  after  the  text  block. 

c.  Option  3:  Clicking  the  button  allows  you  to  specify  where  on  the  page  you  would 
like  to  insert  the  image  by  selecting  the  blocks  (text  or  image)  that  it  should  be 
placed  between. 

2.  Scenario:  You  are  editing  a  block  of  text  and  go  to  the  search  feature.  While  searching  for 
text,  you  come  across  an  image  that  you  would  like  to  add  to  your  page.  Where  on  the 
page  should  the  image  be  inserted? 

a.  Option  1 :  At  the  end  of  the  page. 

b.  Option  2:  After  the  current  text  block. 

c.  Option  3:  A  window  appears  asking  you  to  specify  where  you  want  to  insert  the 
image  by  selecting  the  blocks  that  it  should  be  placed  between. 

3.  Scenario:  You  want  to  insert  a  page  into  the  course. 

a.  You  go  to  the  course  menu  and  click  an  “Edit  Menu”  button. 

i.  Option  1 :  Once  on  the  Edit  Menu  page,  you  click  an  insert  page  button, 
which  takes  you  to  a  page  where  you  specify  the  type  of  page,  the  name  of 
the  page,  and  the  section  into  which  you  want  to  insert  it. 

ii.  Option  2:  Once  on  the  Edit  Menu  page,  there  are  icons  next  to  each 
section  header  that  allow  you  to  insert  a  page  into  that  section.  Once  the 
button  is  clicked,  a  small  pop-up  appears  that  asks  to  choose  a  page  type 
and  give  the  page  a  name. 

4.  Scenario:  You  want  to  rename  a  page  in  the  course. 

a.  You  go  to  the  course  menu  and  click  an  “Edit  Menu”  button. 

i.  Option  1 :  Once  on  the  Edit  Menu  page,  the  menu  is  displayed  again,  but 
each  page  title  is  an  editable  text  box.  You  click  and  type  to  rename. 

ii.  Option  2:  Once  on  the  Edit  Menu  page,  there  is  an  icon  next  to  each  page 
name.  When  you  click  the  icon,  a  small  pop-up  appears  asking  you  to 
rename  the  page. 

5.  Scenario:  You  want  to  reorder  pages  in  a  course. 

a.  You  go  to  the  course  menu  and  click  an  “Edit  Menu”  button. 

i.  Option  1 :  Once  on  the  edit  menu  page,  the  menu  is  displayed  and  the  titles 
are  draggable.  You  drag  and  drop  to  reorder  pages. 

ii.  Option  2:  Once  on  the  Edit  Menu  page,  there  are  arrow  icons  next  to  each 
page  title.  Clicking  the  icons  moves  the  corresponding  page  up  or  down 
one  spot. 
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6.  Scenario:  You  want  to  change  the  layout  of  a  page  (ex:  You  want  the  image  to  be  on  the 
left  of  the  text  instead  of  the  right). 

a.  Option  1 :  When  editing  a  page,  there  is  a  “Change  Page  Layout”  button  that 
allows  you  to  select  from  a  pre-defined  set  of  layouts  based  on  the  type  of  content 
on  your  page. 

b.  Option  2:  When  editing  a  page,  you  can  click  a  button  to  go  into  “Layout  Mode,” 
which  allows  you  to  drag  and  drop  the  pieces  of  content  to  the  locations  that  you 
want  them. 

7.  Scenario:  You  change  a  page  and  save  it,  but  then  decide  you  want  to  undo  what  you  did. 

a.  Option  1 :  Open  the  edit  functionality  and  click  a  “Reset”  button  to  revert  to 
previous  page. 

b.  Option  2:  Have  a  flexible  “Undo”  function  in  the  edit  page. 

8.  Scenario:  You’ve  already  created  a  course  and  want  to  add  a  learning  objective. 

a.  Option  1 :  Create  a  new  course  using  old  course  content  as  generic  content,  where 
appropriate. 

b.  Option  2:  Add  a  learning  objective  to  the  existing  course. 

9.  Scenario:  You  want  to  create  a  course  for  a  new  task 

a.  Option  1 :  Name  new  task  and  “attach”  existing  LOs 

b.  Option  2:  Select  existing  LOs  and  “attach”  a  new  task  name 

10.  Scenario:  You  want  to  create  course  for  a  new  learning  objective 

a.  Option  1 :  Name  new  LO  and  “attach”  an  existing  task 

b.  Option  2:  Select  existing  task  and  “attach”  new  LO 

c.  Options  need  to  include  methods  for  selecting  the  interactivity  template 

11.  Scenario:  You  want  to  create  a  course  for  a  new  task  and  new  learning  objective 
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Appendix  H 

Fort  Carson  Training  Completion  Protocol 


OPTION#! 

1 .  Login  to  the  Army  TA  using  your  username  and  password 

2.  Select  “Route  Security:  Conduct  a  Map  Reconnaissance”  training 

3.  Review  Introduction 

4.  Review  Basic  Tutorial  -  Map  Recon 

5.  Review  What  Right  Looks  Like  -  Map  Recon 

6.  Review  Modified  Approaches  -  Map  Recon 

7.  Complete  Practical  Exercise  -  Map  Recon 

OPTION  #2 

1 .  Login  to  the  Army  TA  using  your  username  and  password 

2.  Select  “Route  Security:  Coordinate  with  Adjacent  Units”  training 

3.  Review  Introduction 

4.  Review  Basic  Tutorial  -  Coordinate  w/Adjacent  Units 

5.  Review  What  Right  Looks  Like  -  Coordinate  w/Adjacent  Units 

6.  Review  Modified  Approaches  -  Coordinate  w/Adjacent  Units 

7.  Complete  Practical  Exercise  -  Coordinate  w/Adjacent  Units 

OPTION  #3 

1 .  Login  to  the  Army  TA  using  your  username  and  password 

2.  Select  “Building  Assault:  Indirect  Fires”  training 

3.  Review  Introduction 

4.  Review  Basic  Tutorial  -  Determine  the  purpose  and  effect  of  indirect  fires 

5.  Review  What  Right  Looks  Like  -  Determine  the  purpose  and  effect  of  indirect  fires 

6.  Review  Modified  Approaches  -  Determine  the  purpose  and  effect  of  indirect  fires 

7.  Complete  Practical  Exercise  -  Determine  the  purpose  and  effect  of  indirect  fires 
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Appendix  I 

Fort  Carson  Training  Generation  Protocol 


Overlay  Creation 

Now  that  you  have  had  an  opportunity  to  review  the  kind  of  product  the  Anny  TA  can  produce, 
we  are  asking  you  to  create  an  exercise  for  yourself.  To  generate  this  exercise,  follow  the 
instructions  given  below.  Note  any  problems  you  encounter  and  raise  your  hand  if  you  need 
assistance. 


1 .  Be  sure  Internet  Explorer  is  set  to  allow  pop-ups. 

2.  Login  to  the  Army  TA  using  the  word  written  at  the  top  of  your  observation  sheet  as  both 
the  username  and  password. 

3.  Select  “Create  New  Training.” 

4.  Select  “Develop  Training  for  a  Doctrinal  Task.” 

5.  Select  “Route  Security  (or  IED  Patrol).” 

6.  Select  “Plan  a  Mission  -  Conduct  a  Map  Reconnaissance.” 

7.  Click  the  “Submit”  button  and  save  the  training. 

8.  Rename  the  training  to  “(Your  Last  Name)  Route  Security  Training.” 

9.  Select  renamed  training. 

10.  Edit  the  text  on  the  Introduction  page  by  doing  the  following: 

a.  Highlight  the  first  sentence  in  yellow. 

b.  Change  one  of  the  headings  to  your  last  name. 

c.  Add  a  link  to  http://www.google.com. 

d.  Change  the  numbered  item  to  a  bulleted  item. 

1 1 .  Edit  a  page  in  the  Basic  Tutorial  by  replacing  the  image  with  an  image  found  via  the 
built-in  search. 

e.  Add  a  caption  to  the  image. 

12.  Edit  a  page  in  the  Basic  Tutorial  by  changing  the  size  of  a  photo. 

13.  Edit  the  Practical  Exercise  Map  Overlay  Creation  Page  as  follows: 

f.  Search  for  a  map  and  insert  it  into  the  background  of  the  overlay. 

g.  Edit  the  “Base  Overlay”  that  students  will  see  as  they  begin  the  practical  exercise: 

i.  Draw  a  route  in  black. 

ii.  Add  a  rally  point  and  label  it  “1.” 

iii.  Add  a  bridge. 

iv.  Add  a  friendly  infantry  platoon. 

v.  Add  a  hostile  signals  intelligence  squad. 

vi.  Add  a  text  box  to  the  map  that  says  “Base  Overlay.”  Make  the  text  green 
and  30  point. 

h.  Lock  the  base  overlay  layer  so  that  students  will  not  be  able  to  edit  it. 

i.  Edit  the  “Answer  Overlay”  map: 

i.  Copy  the  base  overlay  to  the  answer  overlay. 

ii.  Add  a  checkpoint  and  label  it  “1.” 

iii.  Add  an  observation  post. 

iv.  Add  a  friendly  psychological  operations  unit. 
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j .  Save  the  question. 

14.  Make  or  suggest  content  edits  as  they  are  noticed. 

15.  Copy  the  course  you  just  created. 

16.  Delete  the  course  you  just  copied. 

17.  Try  to  recover  the  course  you  just  deleted. 

Test  Question 

Now  that  you  have  had  an  opportunity  to  review  the  kind  of  product  the  Anny  TA  can  produce, 
we  are  asking  you  to  create  an  exercise  for  yourself.  To  generate  this  exercise,  follow  the 
instructions  given  below.  Note  any  problems  you  encounter  and  raise  your  hand  if  you  need 
assistance. 

1 .  Be  sure  Internet  Explorer  is  set  to  allow  pop-ups. 

2.  Login  to  the  Army  TA  using  the  word  written  at  the  top  of  your  observation  sheet  as  both 
the  username  and  password. 

3.  Select  “Create  New  Training.” 

4.  Select  “Develop  Training  for  a  Doctrinal  Task.” 

5.  Select  “Building  Assault.” 

6.  Select  “Plan  the  Employment  of  Fire  Support  -  Detennine  the  Purpose  and  Effect  of 
Indirect  Fires.” 

7.  Click  the  “Submit”  button  and  save  the  training. 

8.  Rename  the  training  to  “(Your  Last  Name)  Fire  Support  Training.” 

9.  Select  the  renamed  training. 

10.  Edit  the  text  on  the  Introduction  page  by  doing  the  following: 

a.  Highlight  one  of  the  sentences  in  blue. 

b.  Using  the  built-in  search  feature,  do  a  search  on  the  tenn  “building  assault.”  Insert 
relevant  text  from  one  of  the  pages  you  find  in  the  search. 

c.  Add  a  link  to  http://www.google.com. 

1 1 .  Edit  the  What  Right  Looks  Like  -  Scenario  page  by  replacing  the  image  with  an  image 
found  via  the  built-in  search. 

a.  Add  a  caption  to  the  image. 

12.  Edit  a  page  in  the  Basic  Tutorial  by  changing  the  size  of  a  photo. 

13.  Edit  the  Practical  Exercise  by  modifying  Question  1 : 

a.  Change  the  question  to  a  true/false  question. 

b.  Indicate  false  as  the  correct  answer. 

c.  Add  an  image  to  the  correct  feedback. 

d.  Save  the  question. 

14.  Make  or  suggest  content  edits  as  they  are  noticed. 

15.  Copy  the  course  you  just  created. 

16.  Delete  the  course  you  just  copied. 
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17.  Try  to  recover  the  course  you  just  deleted. 

Self-Evaluation 

Now  that  you  have  had  an  opportunity  to  review  the  kind  of  product  the  Anny  TA  can  produce, 
we  are  asking  you  to  create  an  exercise  for  yourself.  To  generate  this  exercise,  follow  the 
instructions  given  below.  Note  any  problems  you  encounter  and  raise  your  hand  if  you  need 
assistance. 

1 .  Be  sure  Internet  Explorer  is  set  to  allow  pop-ups. 

2.  Login  to  the  Army  TA  using  the  word  written  at  the  top  of  your  observation  sheet  as  both 
the  username  and  password. 

3.  Select  “Create  New  Training.” 

4.  Select  “Develop  Training  for  a  Doctrinal  Task.” 

5.  Select  “Route  Security  (or  IED  Patrol).” 

6.  Select  “Plan  a  Mission  -  Coordinate  with  Adjacent  Units.” 

7.  Click  the  “Submit”  button  and  save  the  training. 

8.  Rename  the  training  to  “(Your  Last  Name)  Route  Security  Training.” 

9.  Select  the  renamed  training. 

10.  Edit  the  text  on  the  Introduction  page  by  doing  the  following: 

a.  Change  one  of  the  headings  to  your  last  name. 

b.  Using  the  built-in  search  feature,  do  a  search  on  the  tenn  “route  security.”  Insert 
relevant  text  from  one  of  the  pages  you  find  in  the  search. 

c.  Add  a  link  to  http://www.google.com. 

1 1 .  Edit  a  page  in  the  What  Right  Looks  Like  section  by  replacing  the  image  with  an  image 
found  via  the  built-in  search. 

a.  Add  a  caption  to  the  image. 

12.  Edit  a  page  in  the  Basic  Tutorial  by  changing  the  size  of  a  photo. 

13.  Edit  the  Practical  Exercise  by  modifying  the  Self  Evaluation  feedback: 

a.  Change  the  bulleted  list  to  a  numbered  list. 

b.  Delete  the  last  item  on  the  list. 

c.  Change  the  text  color  to  green. 

d.  Bold  the  first  sentence. 

e.  Add  an  image  to  the  feedback. 

f.  Save  the  question. 

14.  Make  or  suggest  content  edits  as  they  are  noticed. 

15.  Copy  the  course  you  just  created. 

16.  Delete  the  course  you  just  copied. 

17.  Try  to  recover  the  course  you  just  deleted. 


1-3 


Appendix  J 

User  Impressions  Survey  Results 


Usability 

Tables  1-3  below  show  the  percentage  of  test  participants  who  endorsed  each  rating  (very  easy 
very  difficult)  on  the  TA  usability  questions.  Table  1  shows  the  breakdown  for  all 
participants.  Figures  1-6  show  the  associated  pie  charts  for  each  question.  Table  2  shows  the 
breakdown  for  non-commissioned  officers  (NCOs)  only.  Table  3  shows  the  breakdown  for 
commissioned  officers  (COs)  only. 

Following  these  tables,  Figures  7-12  graphically  depict  the  NCO/CO  ratings  breakdown  for  each 
usability  question. 


Table  1  -  Usability  Ratings,  All  Test  Participants 


Question1 

Very 

Easy 

Easy 

Neither 

Difficult 

Very 

Difficult 

1 .  How  easy  was  it  to  take  an  Army 
TA  training  package?2 

15% 

70% 

5% 

10% 

0% 

2.  How  easy  was  it  to  construct  a 
training  package  using  the  Anny 

TA? 

11% 

60% 

14% 

11% 

4% 

3.  How  easy  was  it  to  edit  text? 

17% 

63% 

14% 

3% 

3% 

4.  How  easy  was  it  to  use  the  search 
function? 3 

12% 

53% 

21% 

12% 

2% 

5.  How  easy  was  it  to  insert 
images? 

20% 

66% 

3% 

11% 

0% 

6.  How  easy  was  it  to  use  the 
drawing  tools  (if  applicable)?4 

0% 

46% 

40% 

7% 

7% 

rN  =  35  unless  specified  otherwise 
2 A  =  20  for  this  question 
3N=  34  for  this  question 
4N  =  1 5  for  this  question 
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Table  2  -  Usability  Rat 

ings,  NCOs  Only 

Question1 

Very 

Easy 

Easy 

Neither 

Difficult 

Very 

Difficult 

1 .  How  easy  was  it  to  take  an  Army 

2 

TA  training  package? 

0% 

80% 

0% 

20% 

0% 

2.  How  easy  was  it  to  construct  a 
training  package  using  the  Army  TA? 

22% 

34% 

22% 

22% 

0% 

3.  How  easy  was  it  to  edit  text? 

33% 

33% 

22% 

0% 

12% 

4.  How  easy  was  it  to  use  the  search 
function? 

22% 

22% 

11% 

34% 

11% 

5.  How  easy  was  it  to  insert  images? 

44% 

33% 

0% 

23% 

0% 

6.  How  easy  was  it  to  use  the 
drawing  tools  (if  applicable)? 

33% 

33% 

33% 

0% 

0% 

lN=  9  unless  specified  otherwise 
2N=  5  for  this  question 

3 

N  =  6  for  this  question 


Table  3  -  Usability  Ratings,  COs  Only 


Question1 

Very 

Easy 

Easy 

Neither 

Difficult 

Very 

Difficult 

1 .  How  easy  was  it  to  take  an  Army 

2 

TA  training  package? 

20% 

66% 

7% 

7% 

0% 

2.  How  easy  was  it  to  construct  a 
training  package  using  the  Anny  TA? 

8% 

69% 

12% 

8% 

3% 

3.  How  easy  was  it  to  edit  text? 

12% 

73% 

12% 

3% 

0% 

4.  How  easy  was  it  to  use  the  search 
function?3 

8% 

64% 

24% 

4% 

0% 

5.  How  easy  was  it  to  insert  images? 

12% 

77% 

4% 

7% 

0% 

6.  How  easy  was  it  to  use  the  drawing 
tools  (if  applicable)?4 

0% 

33% 

33% 

12% 

22% 

'/V  =  26  unless  specified  otherwise 

2N  =  1 5  for  this  question 

3 

N=  25  for  this  question 

4N=  9  for  this  question 

Army  TA  features  that  posed  a  difficulty. 

•  All  of  my  problems  may  have  been  related  to  the  speed  of  the  computer,  but  the  program 
did  not  seem  to  respond  to  my  input. 

•  Provide  an  edit  option  to  the  line  (i.e.,  Add  points,  delete  points,  move  points)  2)  Fix  the 
unit/symbol  feature.  Automatically  inserts  last  item  and  current  item. 

•  The  drawing  tool  was  very  slow  and  not  very  easy  to  use.  Also,  it  could  use  more 
graphical  control  measures  from  FM  101-5-1  (FM  1-02) 

•  Drawing  tool  has  a  slight  delay.  It  should  be  more  free-flowing  instead  of  straight  angled. 

•  Overlay  products  were  difficult  to  manipulate  (location,  size)  not  intuitive  to  create 
overlays.  Maps  were  difficult  to  read/use  Zoom  function  would've  been  great. 
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•  Map  overlays  had  some  difficulty  loading.  Searches  at  times  froze.  Selecting  certain 
icons  on  map/mo ving/rotating  same  icons 

•  The  issues  were  the  inserting?  Tactical  units  with  the  right  symbols/text  on  the  map 
overlay. 

•  Using  all  of  the  tools  in  the  map  function  while  trying  to  do  an  overlay 

•  First  time  users  of  the  map  functions  and  editions  could  find  is  difficult,  however  once 
it’s  understood  it  is  very  simple. 

•  No  'undo'  feature.  Search  engine  was  sluggish. 

•  The  buttons  in  the  tutorial  and  the  edit  pages  are  not  easily  identifiable  the  edit  text  box  is 
slow  and  cumbersome. 

•  Hard  to  modify  outline  and  basic  structure  freezes  up  during  editing.  Very  rigid  training 
module 

•  Editing  text  and  saving  it 

•  Changing  the  questions 

•  It  was  kinds  hard  to  find  some  buttons  maybe  if  there  was  a  central  location 

•  The  search  would  be  better  as  a  simple  Google  image  search.  I  didn’t  find  the  pictures  I 
wanted. 

•  The  edit  function  seemed  a  bit  awkward  at  times  considering  there  is  the  edit  button  then 
the  edit  icon  a  little  confusing. 

•  Icons  not  showing  their  functionality. 

•  If  you  select  the  pan  Tool  you  lose  your  tool  bar.  Maybe  make  an  option  on  the  tool  bar 
to  pan  w/o  losing  the  tool  bar. 

•  Edit  text  on  map  overlays  -  we  are  used  to  PowerPoint  interface  if  all  map  overlay  tools 
were  modeled  based  on  that  it  would  be  easier. 

•  No  copy-paste  function.  Need  line  point  editing  function.  Need  doctrinally  correct 
OPORD  (Higher)  to  drive  planning.  Need  size  of  icons  to  be  adjustable. 

•  Would  be  easier  on  users  to  add  a  'Microsoft  feel'  to  the  program.  Users  would  be  used  to 
dealing  with  the  Microsoft-like  programs  and  so  it  would  be  easier  to  navigate  from  area 
to  area. 

•  Some  of  the  edit  location  [finding  the  edit  button/icon] 


Utility 

1.  How  useful  do  you  think  the  Army  TA  will  be  for  conducting  unit  training?  (N=  36) 


Definitely  = 

22%  (N  = 

Probably  = 

25%  (N  = 

Maybe  = 

28%  (N  = 

Probably  Not  = 

22%  (N  = 

Definitely  Not  = 

3%  (N  = 

8) 

9) 

10) 
8) 
1) 


Please  list  the  reasons  why/why  not. 


J-3 


•  It  cannot  replace  training  conducted  with  live  human  beings. 

•  It  has  to  include  current  TTP  (which  are  classified) 

•  This  may  be  good  to  train  Platoon  Leaders  on  developing  OPORD  &  graphics,  however  I 
do  not  see  the  utility  in  training  an  infantry  Company  with  a  computer  based  tool. 

•  Some  Soldiers  will  tend  to  skim  thru  the  training.  With  the  overload  of  online 
assessments  and  courses,  the  Soldiers  will  probably  prefer  more  hands-on  training. 

•  Must  be  easier  to  use  than  PowerPoint  which  is  what  most  units  are  using  to  (??)  classes, 
etc. 

•  All  base  infonnation  would  be  standard  Army-wide.  Units  could  see  training  developed 
by  other  units  with  more/different  experience. 

•  Biggest  concern  is  not  enough  computers  with  Internet  access  to  train  on  at  the  unit.  Next 
most  Soldier  tasks  require  hands-on  repetition  for  proficiency  which  does  not  come  from 
computer  training. 

•  Computer  constraints  for  Soldiers  (access).  Time  consuming. 

•  The  more  courses  and  info  that  is  available  will  be  great.  The  reason  it  will  have  trouble 
is  that  there  are  not  enough  computers  at  lower  levels  for  every  Soldier  to  do  this  training. 

•  Easy  to  use  easy  to  access. 

•  The  dynamics  of  [unreadable].  Missions  have  multiple  answers  and  are  often  subjective. 
The  answers  given  by  the  testee  may  be  correct?  But  may  no  coincide  with  the  final 
assessment. 

•  I  won't  train  subordinates  with  one  method  and  evaluate  them  with  another. 

•  Limited  use,  lack  of  computers,  lack  of  practicality.  Might  be  helpful  in  AIT/OBC 

•  The  only  draw  will  be  availability  of  computers.  Other  than  that  a  very  good  tool. 

•  If  it  worked  better  it  could  be  useful 

•  It  gives  leaders  a  base  to  start  training 

•  Lack  of  computers/Soldiers  view  online  training  as  something  they  must  rush 
through/most  units  try  to  do  hands-on  training  on  small  group  discussion,  never 
computers. 

•  Although  there  is  a  computer  shortage  in  most  units  that  will  inhibit  mass  amounts  of 
Soldiers  processing  though  training 

•  It  is  useful  it  creates  redundancy?  A  Soldier  can  train  on  the  computer,  then  go  and  do  it 
the  field. 

•  Maps  preloaded,  less  materials  needed. 

•  Using  the  computer  requires  less  resources  (maps,  pencils,  etc.) 

•  There  are  a  lot  of  software  tools  to  assist  in  planning  training  and  or  missions  for  units. 

It's  really  dependent  on  the  commander  on  what  tool  they  prefer. 

•  Menus  are  intuitive  and  easy  to  understand.  Content  will  make  or  break  this  training  tool. 

•  I  (battalion  level)  do  not  have  time  to  create/alter  courses.  If  there  was  a  database  on 
deployment  specific  tasks  already  available  -  focused  on  platoon/company  level,  that 
would  be  useful. 

•  Good  for  leader  training  &  small  unit  training 

•  As  long  as  the  technology  is  available  and  easy  to  use,  units  will  be  able  to  put  it  to  use. 

•  This  system  would  be  better  suited  to  train  leaders  at  PLDC,  BNOC,  and  WLC  to 
understand  theories  behind  the  practical.  At  unit  level  there  are  too  many  constraints 
time,  equipment,  #  of  personnel  to  train,  individual  computer  understanding. 
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•  Very  easy  to  access  information,  all  at  your  fingertips.  Did  not  have  to  go  into  a  different 
program  IE,  search  engine,  file,  program. 

•  The  TA  system  will  be  good  to  reference  training  and  training  units.  The  TA  system  will 
not  be  good  if  it  is  the  method  of  training. 

•  Who  will  load  images?  Where  will  they  come  from? 

•  It  will  depend  on  Commander. 

•  Have  a  block  of  instruction  at  PLDC  &  BNCOC  to  show  new  small  unit  leaders  what  it  is 
capable  of. 

•  Due  to  time  constraints  and  or  resources  this  TA  can  be  very  helpful. 

2.  Could  you  use  the  Anny  TA  to  create  the  kind  of  training  your  unit  needs?  (N=  36) 

No  =  25%  (N-- =  9) 

Yes  =  75%  (A  =27) 

Please  specify  why/why  not. 

•  The  training  that  I  cannot  depends  on  having  real  Soldiers  will  real  equipment, 
functioning  in  concert  w/each  other. 

•  I  would  use  it  to  help  develop  platoon  leaders,  squad  leaders,  and  Section  SGTs 

•  Again,  this  would  only  be  useful  to  train  leaders  on  planning  an  operation. 

•  It  might  help  for  leaders  to  plan  their  training  and  create  handouts  for  Soldiers  as  a 
reference.  As  far  as  using  this  as  a  primary  trainer,  no  way. 

•  Provided  it  develops  a  robust  database  of  courses,  it  could  be  a  useful  tool. 

•  It  could  be  used  mainly  as  a  leadership  tool.  Average  Soldiers  would  not  require  this  type 
of  training. 

•  Again  my  unit  is  an  artillery  unit  that  most  tasks  require  hands-on  repetition  to  complete 
to  become  proficient  at. 

•  Leaders  could  use  it  to  create  a  class 

•  Our  unit  will  be  conducting  a  non  standard  mission,  being  able  to  find  other  branches 
courses  would  be  very  valuable. 

•  It  has  most  functions  needed  to  do  anything  I  need.  It  would  have  been  nice  to  be  able  to 
see  how  well  the  search  function  worked. 

•  It  does  provide  some  assistance  for  training  in  basic  doctrine. 

•  It  could  be  useful  but  needs  to  be  more  embedded  w/doctrine  and  FMs 

•  Is  geared  toward  leader  training  lack  of  computers/Intemet  in  co  area,  structure  is  too 
rigid  to  easily  modify,  involves  rote  memorization,  not  multiple  scenarios 

•  You  have  the  availability  of  creating  any  class  type  you  need. 

•  I  wasn't  able  to  edit  it  and  save  changes 

•  Give  me  a  start  on  the  work/plus  changing  it  to  my  needs  was  not  hard. 

•  I  think  that  face  to  face  hand  on  combat  training  is  best.  I  see  little  use  for  computer 
based  training  for  my  MOS  that  is. 

•  Collective/individual  task,  Senior  NCO  friendly. 
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•  My  unit  does  not  have  the  computer  resources  and  my  Soldiers  would  not  be  receptive.  I 
think  they  need  trainers  and  instructions  available  to  pass  on  wisdom  and  answer 
questions. 

•  Easy  to  charge  based  on  training  requirements 

•  I  could  use  this  to  conduct  battery  level  training  on  something  like  drivers  or  range  safety 
certification. 

•  Once  the  programs  are  in  the  computer  Soldiers  of  today  play  games  with  computers  and 
they  are  so  similar  with  their  function  it  will  make  it  interesting  than  from  your  traditional 
black  board. 

•  Map  reading  is  always  useful. 

•  Most  of  the  training  my  Soldiers  need  is  hands-on  and  is  best  learned  with  the  equipment 
in  hand. 

•  This  is  a  good  training  application  for  higher  level  HQ  to  create  scenarios  for  subordinate 
units.  Time  intensive  for  Battalion  and  below. 

•  But  based  on  current  ARFORGEN/deployment  timeline  I  (my  unit)  does  not  have  the 
time/manpower  to  create  training  -  could  tailor  already  existing  training. 

•  Not  applicable  at  Corps  level,  we  use  Command  Post  of  the  Future  &  large  scale 
simulations. 

•  It  would  help  in  making  scenarios  based  training  to  make  Soldiers  use  their  heads  and 
think  on  their  feet  as  opposed  to  cut  and  dry  doctrine. 

•  Excellent  way  of  transferring  SOPs  but  there  are  constraints  at  unit  level. 

•  It  is  customized  to  my  needs.  PowerPoint,  Word,  Excel,  all  rolled  into  one. 

•  The  TA  will  create  a  good  source  for  our  unit  to  rotate  training  both  hands-on  and  online. 

•  It  can  be  tailored  to  unit  need. 

•  This  can  be  used  for  refresher  training  or  for  extra  training. 

3.  Would  you  use  the  Army  TA  to  create  unit  training?  (A  =  35) 

No  =  34%  (A  =12) 

Yes  =  66%  (A  =23) 

Challenges  to  using  the  Army  TA  for  conducting  unit  training: 

•  Must  have  real  one-on-one  Soldier  interaction. 

•  Classified  material/accessibility. 

•  Lack  of  computers  to  train  individuals.  2)  Not  geared  towards  individual  tasks. 

•  Getting  seasoned  senior  NCOs  who  have  developed  their  own  TTP  and  SOP  to  adopt  it. 

•  Access  to  database  from  theater  (opportunity  &  speed)  Resident  knowledge  to  solve 
problems. 

•  Computer  access/network  reliability.  Not  pertinent  training  for  all  Soldiers 

•  Biggest  issue  will  be  having  enough  computers  with  LAN  access  at  the  unit  level  for 
training. 

•  I  think  it  would  be  easier  to  train  in  a  field  environment. 

•  Not  enough  access  to  computers  so  that  every  Soldier  can  do  the  practical  exercise 
portion. 
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•  Time  and  Internet  constrains  in  while  deployed.  It's  quick  but  [unreadable]  Internet  could 
make  it  more  difficult. 

•  Prefer  using  group  discussions  (AAR)  as  opposed  this  'Stove  Pipe’  method. 

•  No  computers.  Best  training  method  is  a  good  NCO. 

•  See  previous  answer.  Most  training  is  either  hands-on  or  based  on  multiple  scenarios. 

•  Too  many  Soldiers  and  not  enough  computers. 

•  Can't  be  taken  to  the  field/Changing  it  to  Battalion  guidelines. 

•  Needs  more  info  i.e.,  References,  FMs,  Books,  points  of  contact. 

•  Resources.  No  'teacher'  available  to  Soldiers. 

•  Possibly  accessing  it. 

•  Will  keep  Soldiers  more  focus  on  the  learning  objective  than  trying  to  teach  on  black 
board. 

•  Getting  laptops  for  the  platoon  would  be  the  hardest  part. 

•  Learning  curve  for  the  Soldiers  using  the  software. 

•  Time  to  create  relative  scenarios. 

•  time/manpower. 

•  See  observer  notes.  Not  Blue  Force  Tracker,  Command  Post  of  the  Future,  FBCB2. 

•  Availability,  user  friendly. 

•  Some  area  of  training  require  more  hands-on  than  actual  reading. 

•  Very  user  friendly;  no  challenges  if  you  know  what  you  are  trying  to  accomplish. 

•  Will  need  to  have  enough  computers  and  [no(?)]  time  constraints. 

•  It  will  take  time  to  leam  interface  make  it  more  like  Microsoft  office  products  and  people 
will  learn  it  faster. 

•  Time-consuming  for  the  instructor  to  set  training  up,  however,  once  established,  the 
training  would  be  very  useful. 

Feasible  alternatives  to  using  the  Army  TA  for  conducting  unit  training: 

•  The  old  fashioned  hands  on  training  we  all  know  and  love. 

•  Non-Commissioned  Officer  Professional  Development  and  Officer  Professional 
Development. 

•  Situational  Field  Training  Exercises  2)  Line  fire  exercises. 

•  More  hands-on  field  training. 

•  Nothing  beats  hands-on,  feet-to-the-fire  training.  Close  Combat  Tactical  Trainer  provides 
similar  training  -  use  Army  TA  in  conjunction  with  Close  Combat  Tactical  Trainer  and 
other  simulations. 

•  Classroom  training  with  a  projector,  field  training,  hands  on  individual  training. 

•  Having  pictures  or  videos  of  what  the  right  way  to  do  training  is,  so  instead  of  going  step 
by  step  over  slides,  classes  are  also  on  a  video  link  explaining  the  text. 

•  PowerPoint. 

•  Would  like  to  see  a  system  that  would  consolidate  several  users  into  group  discussions. 

•  Using  Blue  Force  Tracker/FBCB2. 

•  Video  games.  Field  training.  Manuals/doctrine. 

•  Small  group  briefings. 
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•  Putting  the  classes  on  paper  and  training  a  unit  call  at  once. 

•  AMs/Books/Training  Manuals. 

•  Squad  leader,  team  leader. 

•  FM  and  field  training.  Field  manual. 

•  Hands-on  training/chalk  board  training/computer  based  Advanced  Field  Artillery  Tactical 
Data  System  (classified).  This  is  great  for  the  visual  learner. 

•  Hard  copy  tests/training. 

•  Old  fashioned  paper  maps. 

•  PowerPoint  briefing,  small  unit  groups. 

•  Existing  Army  systems  combined  with  written  scenarios  can  achieve  the  same  desired 
result. 

•  Products  that  currently  exist  -  PowerPoint  briefings/online  courses  offered  on  Army 
Knowledge  Online. 

•  Leader  training  &  small  unit  training. 

•  Unit's  already  using  hands-on  training  that  is  provided.  This  may  be  a  good  alternative. 

•  Training  rotation  with  other  units  to  do  both  hands-on  and  reference  the  training  online. 

•  Skill  port/Skill  soft 


Effectiveness 

How  effective  do  you  think  the  training  generated  by  the  Army  TA  can  be  for  addressing  key 
training  requirements  in  your  unit?  (N=  35) 


Very  Effective  = 
Somewhat  Effective  = 
Neither  = 

Somewhat  Ineffective  = 
Very  Ineffective  = 


17%  (N=  6) 
66%  (N=  23) 
11%  (N=  4) 
3%  (N=  1) 

3  %(N=  1) 


Training  requirements  you  think  the  Army  TA  training  could  address  effectively. 


•  Mission  Planning. 

•  Mission  planning  2)  Operational  Planning. 

•  Drawing  overlays. 

•  Any  tactical  scenario  (cordon  &  search  ambush,  room  clearing). 

•  Leadership  development  from  squad  leader  up.  Decision-making  skills. 

•  Unit  movement,  Hazmat,  Ammo  handlers  course,  shell  &  fuse  combination  for  field 
artillery  units. 

•  Convoy.  Route  Clearance.  Room  clearing. 

•  Tactical  training  &  other  training  such  as  safety  courses,  emergency  operations,  etc. 

•  It  would  enable  people  to  make  any  kind  of  training.  Anytime.  Even  those  without 
systems  like  PowerPoint. 

•  Information  operations.  ROE. 

•  A  practical  exercise  in  a  classroom  will  not  address  the  necessary  training  requirement  of 
face  to  face  training  &  evaluation. 

•  Basic  doctrine. 
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•  Cordon  and  search,  patrol  base  ops,  casualty  evacuation  procedures  (Urban  and  Mountain 
terrain). 

•  ROE. 

•  Knowing  the  basics/giving  new  Soldiers  a  guide  line. 

•  Training  units  that  don’t  always  do  the  stuff  addressed  in  the  TA  trainer. 

•  This  program  is  good  for  big  Anny  though  brigade  to  create  lessons  and  practical 
exercises  for  safety  modules  or  any  training  they  want  all  Soldiers  to  take. 

•  Range  safety  certification  safety  training.  Drives  training.  Privately-owned  vehicle  risk 
assessment. 

•  Map  reading,  recon,  and  symbols.  Route  recon. 

•  Basic  Soldier  tasks  that  are  unifonn  throughout  the  Anny  such  as  map  reading. 

•  Pre-deployment  specific  tasks. 

•  Leader  training  &  small  unit  training. 

•  This  can  be  better  for  long  term-older  branches  such  as  Infantry/Field  artillery.  I  am  Air 
Defense  and  we  have  our  own  Army  training  technology  to  train  on  for  those  tactics.  This 
would  be  good  for  basic  Army  infantry  skills  (leadership). 

•  Leader  training. 

•  The  system  could  add  more  training  on  combat  operation  as  well  as  MOS  training. 

•  On  the  spot  training  that  is  fast  and  readily  available. 

•  ROE  would  be  a  great  way  to  start. 

General 

1.  What  did  you  particularly  like  about  the  Army  TA? 

•  Address  a  large  audience. 

•  Provides  a  digital  tool  to  train  Platoon  Leaders  on  planning  operations. 

•  Lessons  and  references. 

•  Potential  database  of  courses.  Course  formatting. . .  fill  in  the  blank+good. 

•  Ease  (general)  of  use.  Covering  all  bases  with  the  general  knowledge 

•  The  TA  was  easy  to  use,  modify,  and  was  overall  user  friendly. 

•  Ease  to  use. 

•  Very  easy  to  learn. 

•  Quick  and  easy  also  I  ran  make  a  text  only  class  that  would  be  easier  for  the  slow  Internet 

during  deployment. 

•  User  friendly. 

•  Good  idea. 

•  Very  simple. 

•  Versatility. 

•  The  layouts. 

•  Being  able  to  change  the  course. 

•  It  does  give  you  an  idea  of  what  needs  to  do. 

•  It  was  flexible  and  you  don’t  need  to  start  from  scratch  with  your  training. 

•  Great  userability/very  simple 

•  User-friendly. 
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•  Very  small  and  handy  and  easy  to  maneuver  and  set  up. 

•  Easy  to  use. 

•  It  could  allow  Soldiers  to  take  the  training  at  their  own  pace. 

•  Ease  of  use. 

•  Ability  to  tailor  courses,  ability  for  individual  users  to  access/use  several  different 
courses. 

•  Can  create  entire  programs  of  instruction/classes  tailored  for  leader  development;  good 
for  remote  training. 

•  It  is/can  be  interactive.  A  Soldier  can  do  this  at  home,  on  his  own. 

•  Ease  of  use  and  ability  to  change  to  unit  SOPs. 

•  All  tools/materials  needed  are  at  my  fingertips. 

•  Easy  access  user  friendly. 

•  Easy  to  navigate. 

•  It’s  simple  use. 

•  Fairly  easy  to  use. 

•  Uses  very  similar  buttons  to  common  software. 

•  Simple  to  use.  Straightforward. 

2.  What  did  you  particularly  dislike  about  the  Anny  TA? 

•  Was  not  user-friendly  for  me. 

•  Too  vague. 

•  Not  focused  on  training  a  unit  2)  Difficult  to  use  drawing  interface 

•  Feels  too  much  like  Blue  Force  Tracker,  but  is  easier  to  use. 

•  Overlay  tools. 

•  Difficulty  in  drawing/creating  map  overlays. 

•  I  don’t  like  computer  based  training  because  it  removes  the  Soldier  from  the  outside 
training  environment  with  hands-on  training. 

•  Time  consuming. 

•  Not  easy  to  search  for  help  when  trying  to  figure  out  how  to  do  stuff. 

•  Cannot  look  out  the  ’Next  button’  under  certain  conditions. 

•  Difficult  to  read  and  follow  a  map  when  both  are  not  visible  simultaneously 

•  Not  doctrinally  correct.  Too  civilian/simplified.  Not  intuitive. 

•  Not  really  interactive.  Content  sucks. 

•  Not  practical. 

•  Can’t  be  e-mailed  or  saved  to  other  people. 

•  Skips  over  a  lot  of  stuff.  Needs  more  content. 

•  I  believe  it  is  not  practical  for  company  and  below  training. 

•  The  search  option  could  be  made  simpler 

•  I  didn’t  really  dislike  anything 

•  Not  being  able  to  use  it  at  the  unit  level  -  not  enough  laptops. 

•  Not  sure  how  useful  it  would  be  in  branch/MOS  specific  missions  might  be  good  for 
familiarizing  Soldiers  w/  equipment  but  not  the  functionality. 

•  Relies  heavily  on  scenarios  that  take  time  to  build. 
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•  Amount  of  time  it  takes  to  develop/create  training. 

•  Needs  to  be  linked  to  doctrinal  references  &  more  like  PowerPoint  commands. 

•  Not  as  friendly  to  the  user.  Takes  someone  to  push  buttons  to  teach  themselves  how  to 
navigate  through  the  pages. 

•  Not  understanding  Soldier  constraints  at  small  unit  level. 

•  Some  location  of  edit  button  location  [finding  the  edit  button/icon] 

•  Search  wasn’t  as  intuitive  as  it  could  be. 

3.  What  recommendations  do  you  have  for  improving  the  Army  TA? 

•  Stop  development.  We’re  trying  to  reinvent  the  wheel  here.  The  Army  has  training 
programs  in  place  that  work.  It  ain't  broke  -  don’t  try  to  fix  it! 

•  Base  scenarios  on  doctrine  and  add  TTP  (local  area  and  theater)  for  trainer  options 

•  Improve  drawing  capabilities  and  editing  tools,  no  one  will  use  this  if  it  is  difficult  to 
develop  scenarios  for  training. 

•  Have  an  option  in  the  beginning  to  select  your  type  of  unit  (i.e.  Fire  Support  Cell  mortar 
platoon,  sniper  team,  etc.)  include  METL  for  each  type  of  unit. 

•  Develop,  manage  and  provide  oversight  on  course  database.  Biggest  competitor  is 
Microsoft  PowerPoint,  must  be  same  interface  or  easier. 

•  Make  it  easier  for  the  instructor  to  edit  quickly.  Ensure  courses/training  is  not  limited  by 
site  or  unit  -  all  classes  across  the  Army  should  be  accessible  to  everyone.  Find  a  way  to 
coordinate  use  of  TA  (specifically  route  recons,  etc.)  in  conjunction  with  Close  Combat 
Tactical  Trainer  and  Engagement  Skills  Trainer  2000  and  other  Anny  technology 

•  It  is  a  good  tool  but  should  be  used  for  limited  personnel. 

•  Fix  small  glitches  annotated?  On  note  page,  [unreadable]  The  amount  of  text. 

•  Integrate  tasks,  conditions  and  standards  from  each  MOS  &  Branches  Mission  Essential 
task  list  (METL)  so  it  makes  it  easier  for  leaders  to  find  the  correct  training.  Also 
integrate  CALL  Lessons  learned  to  stay  current  with  new  trends. 

•  Ability  to  lockout  the  ’Next’  button  until  certain  conditions  are  met  on  the  page. 

•  Make  it  more  interactive  (i.e.,  Encyclopedia  programs).  Animations  to  go  with  the 
material,  or  maybe  videos  showing  actual  events  (the  do's  and  don'ts) 

•  Soldiers  need  consistent  training  that  matches  what  they  see  in  combat. 

•  More  interactive.  More  real  world. 

•  Not  an  issue  now  but  once  classes  start  flooding  in  on  a  master  page,  have  them 
categorized  by  combat  missions,  logistic,  personnel,  and  so  on. 

•  Make  it  more  directly  correlate  to  how  we  do  things  not  what  we  do 

•  Printing  option/dictionary. 

•  Have  a  mix  of  both  book  and  personal  ideas. 

•  I  do  not  think  this  project  should  be  pursued. 

•  Have  a  way  to  start  from  scratch.  Be  able  to  print  certificates. 

•  Have  cursor  be  able  to  identify  and  explain  what  each  icon  does. 

•  Consider  renaming  layers.  Different  icon  for  the  'fill'  option.  More  'save'  buttons 
available. 

•  Making  the  images  more  interactive  for  example  create  an  image  map  of  radio,  select  the 
buttons  as  hot  spots  that  would  be  'configurable'  to  allow  the  user  to  input  different 
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frequencies,  channels,  also  display  pop  up  infonnation  that  would  benefit  the  user. 
Interactive  modules  for  treating  a  casualty,  secure  prisoners  of  war. 

•  Manipulating  images/icons  should  mirror  PowerPoint.  Create  a  database  of 
scenarios/programs  of  instruction  or  be  able  to  pull  from  existing  Army  databases. 

•  User  friendly  (Microsoft-like). 

•  Proposed  audience  -  school  house,  unit  leadership  (battalion  &  higher),  train  new  leaders 
on  SOPs  BN  level. 

•  Continue  to  do  survey  to  work  on  some  of  the  defaults  in  the  program  software. 

•  Come  up  with  tutorials  for  the  basic  Army  make  up;  i.e.,  combat  arms,  combat  support, 
combat  service  support,  aviation. 

•  Continue  the  'user  friendly'  testing. 

•  I  think  once  a  user  becomes  familiar  w/the  system  it  will  become  easier  to  use. 
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